task e2e-llm-inference-service has failed: "step-fail-if-needed" exited with code 1: Error [get-kubeconfig] Found kubeconfig secret: cluster-sdbb4-admin-kubeconfig [get-kubeconfig] Wrote kubeconfig to /credentials/cluster-sdbb4-kubeconfig [get-kubeconfig] Found admin password secret: cluster-sdbb4-admin-password [get-kubeconfig] Retrieved username [get-kubeconfig] Wrote password to /credentials/cluster-sdbb4-password [get-kubeconfig] API Server URL: https://a1d2c19a6f55d48ecac1643dacbe58fa-b116a92104c2b9dc.elb.us-east-1.amazonaws.com:6443 [get-kubeconfig] Console URL: https://console-openshift-console.apps.6eeb9ecb-e130-4fe7-acae-0164835f2548.prod.konfluxeaas.com [clone-repo] sync-260708_master-to-release-v0.17 [clone-repo] https://github.com/maskarb/kserve [clone-repo] Cloning into '/workspace/source'... [clone-repo] Updating files: 86% (2688/3118) Updating files: 87% (2713/3118) Updating files: 88% (2744/3118) Updating files: 89% (2776/3118) Updating files: 90% (2807/3118) Updating files: 91% (2838/3118) Updating files: 92% (2869/3118) Updating files: 93% (2900/3118) Updating files: 94% (2931/3118) Updating files: 95% (2963/3118) Updating files: 96% (2994/3118) Updating files: 97% (3025/3118) Updating files: 98% (3056/3118) Updating files: 99% (3087/3118) Updating files: 100% (3118/3118) Updating files: 100% (3118/3118), done. [e2e-llm-inference-service] + bash [e2e-llm-inference-service] + STATUS_FILE=/test-status/deploy-and-e2e-status [e2e-llm-inference-service] + echo failed [e2e-llm-inference-service] + COMPONENT_NAME=kserve-agent-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-agent-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:dfb2082d89c1ab80bec06b84d0a6317b542c29ff4f60aa1cefe62d44619bad8a [e2e-llm-inference-service] + KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:dfb2082d89c1ab80bec06b84d0a6317b542c29ff4f60aa1cefe62d44619bad8a [e2e-llm-inference-service] + COMPONENT_NAME=kserve-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:f494edae70be745f0175d334523db2d34c4b6619a3e2d1bad8c7a26323eef5b3 [e2e-llm-inference-service] + KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:f494edae70be745f0175d334523db2d34c4b6619a3e2d1bad8c7a26323eef5b3 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-router-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-router-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:95154c3f09f98c64e7d79b759f3f17a91001c79b4e6c56ac6d0a6d8b1e47cf38 [e2e-llm-inference-service] + KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:95154c3f09f98c64e7d79b759f3f17a91001c79b4e6c56ac6d0a6d8b1e47cf38 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-storage-initializer-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-storage-initializer-ci '.[$component_name].image' [e2e-llm-inference-service] + export STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e [e2e-llm-inference-service] + STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e [e2e-llm-inference-service] + COMPONENT_NAME=odh-kserve-llmisvc-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name odh-kserve-llmisvc-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:4c9e95dda047d9c923ada8c9d72c4b97eb988a42011479795898b1bc9b478f4f [e2e-llm-inference-service] + LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:4c9e95dda047d9c923ada8c9d72c4b97eb988a42011479795898b1bc9b478f4f [e2e-llm-inference-service] + ./test/scripts/openshift-ci/run-e2e-tests.sh 'llminferenceservice and cluster_cpu and not autoscaling and not tracing' 2 llm-d [e2e-llm-inference-service] INFERENCE_POOL_GROUP=inference.networking.k8s.io (detected from OCP 4.21.23) [e2e-llm-inference-service] Installing on cluster [e2e-llm-inference-service] Using namespace: kserve for KServe components [e2e-llm-inference-service] SKLEARN_IMAGE=quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1711 [e2e-llm-inference-service] OPT_125M_MODEL_URI=s3://example-models/facebook/opt-125m [e2e-llm-inference-service] ERROR_404_ISVC_IMAGE=quay.io/opendatahub/error-404-isvc:odh-pr-1711 [e2e-llm-inference-service] SUCCESS_200_ISVC_IMAGE=quay.io/opendatahub/success-200-isvc:odh-pr-1711 [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed Kustomize v5.8.1 to /workspace/source/bin/kustomize [e2e-llm-inference-service] v5.8.1 [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] [INFO] Installing yq v4.52.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed yq v4.52.1 to /workspace/source/bin/yq [e2e-llm-inference-service] yq (https://github.com/mikefarah/yq/) version v4.52.1 [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Installing KServe Python SDK ... [e2e-llm-inference-service] [INFO] Installing uv 0.7.8 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed uv 0.7.8 to /workspace/source/bin/uv [e2e-llm-inference-service] warning: Failed to read project metadata (No `pyproject.toml` found in current directory or any parent directory). Running `uv self version` for compatibility. This fallback will be removed in the future; pass `--preview` to force an error. [e2e-llm-inference-service] uv 0.7.8 [e2e-llm-inference-service] Creating virtual environment... [e2e-llm-inference-service] warning: virtualenv's `--clear` has no effect (uv always clears the virtual environment) [e2e-llm-inference-service] Using CPython 3.9.25 interpreter at: /usr/bin/python3 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] Using CPython 3.11.13 interpreter at: /usr/bin/python3.11 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] Resolved 266 packages in 1ms [e2e-llm-inference-service] Building kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading grpcio-tools (2.5MiB) [e2e-llm-inference-service] Downloading uvloop (3.8MiB) [e2e-llm-inference-service] Downloading aiohttp (1.7MiB) [e2e-llm-inference-service] Downloading setuptools (1.2MiB) [e2e-llm-inference-service] Downloading grpcio (6.4MiB) [e2e-llm-inference-service] Downloading botocore (12.9MiB) [e2e-llm-inference-service] Downloading pandas (12.5MiB) [e2e-llm-inference-service] Downloading kubernetes (1.9MiB) [e2e-llm-inference-service] Downloading black (1.6MiB) [e2e-llm-inference-service] Downloading pyarrow (40.1MiB) [e2e-llm-inference-service] Downloading numpy (15.7MiB) [e2e-llm-inference-service] Downloading cryptography (4.3MiB) [e2e-llm-inference-service] Downloading pydantic-core (2.0MiB) [e2e-llm-inference-service] Downloading mypy (17.2MiB) [e2e-llm-inference-service] Downloading portforward (3.9MiB) [e2e-llm-inference-service] Building python-simple-logger==2.0.19 [e2e-llm-inference-service] Building timeout-sampler==1.0.3 [e2e-llm-inference-service] Downloading aiohttp [e2e-llm-inference-service] Downloading black [e2e-llm-inference-service] Downloading pydantic-core [e2e-llm-inference-service] Downloading grpcio-tools [e2e-llm-inference-service] Downloading setuptools [e2e-llm-inference-service] Downloading portforward [e2e-llm-inference-service] Built python-simple-logger==2.0.19 [e2e-llm-inference-service] Downloading uvloop [e2e-llm-inference-service] Downloading cryptography [e2e-llm-inference-service] Downloading grpcio [e2e-llm-inference-service] Downloading kubernetes [e2e-llm-inference-service] Built timeout-sampler==1.0.3 [e2e-llm-inference-service] Downloading numpy [e2e-llm-inference-service] Built kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading pandas [e2e-llm-inference-service] Downloading botocore [e2e-llm-inference-service] Downloading pyarrow [e2e-llm-inference-service] Downloading mypy [e2e-llm-inference-service] Prepared 101 packages in 1.86s [e2e-llm-inference-service] warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance. [e2e-llm-inference-service] If the cache and target directories are on different filesystems, hardlinking may not be supported. [e2e-llm-inference-service] If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning. [e2e-llm-inference-service] Installed 101 packages in 364ms [e2e-llm-inference-service] + aiohappyeyeballs==2.6.1 [e2e-llm-inference-service] + aiohttp==3.13.3 [e2e-llm-inference-service] + aiosignal==1.4.0 [e2e-llm-inference-service] + annotated-doc==0.0.4 [e2e-llm-inference-service] + annotated-types==0.7.0 [e2e-llm-inference-service] + anyio==4.9.0 [e2e-llm-inference-service] + attrs==25.3.0 [e2e-llm-inference-service] + avro==1.12.0 [e2e-llm-inference-service] + black==24.3.0 [e2e-llm-inference-service] + boto3==1.37.35 [e2e-llm-inference-service] + botocore==1.37.35 [e2e-llm-inference-service] + cachetools==5.5.2 [e2e-llm-inference-service] + certifi==2025.1.31 [e2e-llm-inference-service] + cffi==2.0.0 [e2e-llm-inference-service] + charset-normalizer==3.4.1 [e2e-llm-inference-service] + click==8.1.8 [e2e-llm-inference-service] + cloudevents==1.11.0 [e2e-llm-inference-service] + colorama==0.4.6 [e2e-llm-inference-service] + colorlog==6.10.1 [e2e-llm-inference-service] + coverage==7.8.0 [e2e-llm-inference-service] + cryptography==46.0.5 [e2e-llm-inference-service] + deprecation==2.1.0 [e2e-llm-inference-service] + durationpy==0.9 [e2e-llm-inference-service] + execnet==2.1.1 [e2e-llm-inference-service] + fastapi==0.136.3 [e2e-llm-inference-service] + frozenlist==1.5.0 [e2e-llm-inference-service] + google-auth==2.39.0 [e2e-llm-inference-service] + grpc-interceptor==0.15.4 [e2e-llm-inference-service] + grpcio==1.78.1 [e2e-llm-inference-service] + grpcio-testing==1.78.1 [e2e-llm-inference-service] + grpcio-tools==1.78.1 [e2e-llm-inference-service] + h11==0.16.0 [e2e-llm-inference-service] + httpcore==1.0.9 [e2e-llm-inference-service] + httptools==0.6.4 [e2e-llm-inference-service] + httpx==0.27.2 [e2e-llm-inference-service] + httpx-retries==0.4.5 [e2e-llm-inference-service] + idna==3.10 [e2e-llm-inference-service] + iniconfig==2.1.0 [e2e-llm-inference-service] + jinja2==3.1.6 [e2e-llm-inference-service] + jmespath==1.0.1 [e2e-llm-inference-service] + kserve==0.19.0 (from file:///workspace/source/python/kserve) [e2e-llm-inference-service] + kubernetes==32.0.1 [e2e-llm-inference-service] + markupsafe==3.0.2 [e2e-llm-inference-service] + multidict==6.4.3 [e2e-llm-inference-service] + mypy==0.991 [e2e-llm-inference-service] + mypy-extensions==1.0.0 [e2e-llm-inference-service] + numpy==2.2.4 [e2e-llm-inference-service] + oauthlib==3.2.2 [e2e-llm-inference-service] + orjson==3.10.16 [e2e-llm-inference-service] + packaging==24.2 [e2e-llm-inference-service] + pandas==2.2.3 [e2e-llm-inference-service] + pathspec==0.12.1 [e2e-llm-inference-service] + platformdirs==4.3.7 [e2e-llm-inference-service] + pluggy==1.5.0 [e2e-llm-inference-service] + portforward==0.7.1 [e2e-llm-inference-service] + prometheus-client==0.21.1 [e2e-llm-inference-service] + propcache==0.3.1 [e2e-llm-inference-service] + protobuf==6.33.5 [e2e-llm-inference-service] + psutil==5.9.8 [e2e-llm-inference-service] + pyarrow==19.0.1 [e2e-llm-inference-service] + pyasn1==0.6.3 [e2e-llm-inference-service] + pyasn1-modules==0.4.2 [e2e-llm-inference-service] + pycparser==2.22 [e2e-llm-inference-service] + pydantic==2.12.4 [e2e-llm-inference-service] + pydantic-core==2.41.5 [e2e-llm-inference-service] + pyjwt==2.12.1 [e2e-llm-inference-service] + pytest==7.4.4 [e2e-llm-inference-service] + pytest-asyncio==0.23.8 [e2e-llm-inference-service] + pytest-cov==5.0.0 [e2e-llm-inference-service] + pytest-httpx==0.30.0 [e2e-llm-inference-service] + pytest-json-report==1.5.0 [e2e-llm-inference-service] + pytest-metadata==3.1.1 [e2e-llm-inference-service] + pytest-xdist==3.6.1 [e2e-llm-inference-service] + python-dateutil==2.9.0.post0 [e2e-llm-inference-service] + python-dotenv==1.1.0 [e2e-llm-inference-service] + python-multipart==0.0.22 [e2e-llm-inference-service] + python-simple-logger==2.0.19 [e2e-llm-inference-service] + pytz==2025.2 [e2e-llm-inference-service] + pyyaml==6.0.2 [e2e-llm-inference-service] + requests==2.32.3 [e2e-llm-inference-service] + requests-oauthlib==2.0.0 [e2e-llm-inference-service] + rsa==4.9.1 [e2e-llm-inference-service] + s3transfer==0.11.4 [e2e-llm-inference-service] + setuptools==78.1.0 [e2e-llm-inference-service] + six==1.17.0 [e2e-llm-inference-service] + sniffio==1.3.1 [e2e-llm-inference-service] + starlette==1.2.1 [e2e-llm-inference-service] + tabulate==0.9.0 [e2e-llm-inference-service] + timeout-sampler==1.0.3 [e2e-llm-inference-service] + timing-asgi==0.3.1 [e2e-llm-inference-service] + tomlkit==0.13.2 [e2e-llm-inference-service] + typing-extensions==4.15.0 [e2e-llm-inference-service] + typing-inspection==0.4.2 [e2e-llm-inference-service] + tzdata==2025.2 [e2e-llm-inference-service] + urllib3==2.6.2 [e2e-llm-inference-service] + uvicorn==0.34.1 [e2e-llm-inference-service] + uvloop==0.21.0 [e2e-llm-inference-service] + watchfiles==1.0.5 [e2e-llm-inference-service] + websocket-client==1.8.0 [e2e-llm-inference-service] + websockets==15.0.1 [e2e-llm-inference-service] + yarl==1.20.0 [e2e-llm-inference-service] Audited 1 package in 48ms [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [INFO] Kustomize v5.8.1 is already installed in /workspace/source/bin (>= v5.8.1) [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Now using project "kserve" on server "https://a1d2c19a6f55d48ecac1643dacbe58fa-b116a92104c2b9dc.elb.us-east-1.amazonaws.com:6443". [e2e-llm-inference-service] [e2e-llm-inference-service] You can add applications to this project with the 'new-app' command. For example, try: [e2e-llm-inference-service] [e2e-llm-inference-service] oc new-app rails-postgresql-example [e2e-llm-inference-service] [e2e-llm-inference-service] to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application: [e2e-llm-inference-service] [e2e-llm-inference-service] kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname [e2e-llm-inference-service] [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [INFO] Kustomize v5.8.1 is already installed in /workspace/source/bin (>= v5.8.1) [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Creating namespace openshift-keda... [e2e-llm-inference-service] namespace/openshift-keda created [e2e-llm-inference-service] Namespace openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating OperatorGroup openshift-keda... [e2e-llm-inference-service] operatorgroup.operators.coreos.com/openshift-keda created [e2e-llm-inference-service] OperatorGroup openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating Subscription for openshift-custom-metrics-autoscaler-operator... [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-custom-metrics-autoscaler-operator created [e2e-llm-inference-service] Subscription openshift-custom-metrics-autoscaler-operator created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Waiting for openshift-custom-metrics-autoscaler-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (15/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (20/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (25/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-1 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-1 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-1 found, but not yet Succeeded (Phase: Installing). Waiting... (40/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-1 found, but not yet Succeeded (Phase: Installing). Waiting... (45/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-1 found, but not yet Succeeded (Phase: Installing). Waiting... (50/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-1 is ready (Phase: Succeeded). [e2e-llm-inference-service] --- [e2e-llm-inference-service] Applying KedaController custom resource... [e2e-llm-inference-service] Warning: resource kedacontrollers/keda is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] kedacontroller.keda.sh/keda configured [e2e-llm-inference-service] KedaController custom resource applied. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Allowing time for KEDA components to be provisioned by the operator ... [e2e-llm-inference-service] Waiting for KEDA Operator pod (selector: "app=keda-operator") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-operator" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-operator" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-operator" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-operator-68bff59c-hp52q 1/1 Running 0 44s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-operator" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-operator-68bff59c-hp52q condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-operator" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Operator pod is ready. [e2e-llm-inference-service] Waiting for KEDA Metrics API Server pod (selector: "app=keda-metrics-apiserver") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-metrics-apiserver" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-metrics-apiserver-559f75f947-686s6 1/1 Running 0 48s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-metrics-apiserver-559f75f947-686s6 condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Metrics API Server pod is ready. [e2e-llm-inference-service] Waiting for KEDA Webhook pod (selector: "app=keda-admission-webhooks") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-admission-webhooks" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-admission-5fbd5c4644-bs4k6 1/1 Running 0 54s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-admission-5fbd5c4644-bs4k6 condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Webhook pod is ready. [e2e-llm-inference-service] --- [e2e-llm-inference-service] ✅ KEDA deployment script finished successfully. [e2e-llm-inference-service] KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:f494edae70be745f0175d334523db2d34c4b6619a3e2d1bad8c7a26323eef5b3 [e2e-llm-inference-service] LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:4c9e95dda047d9c923ada8c9d72c4b97eb988a42011479795898b1bc9b478f4f [e2e-llm-inference-service] KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:dfb2082d89c1ab80bec06b84d0a6317b542c29ff4f60aa1cefe62d44619bad8a [e2e-llm-inference-service] KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:95154c3f09f98c64e7d79b759f3f17a91001c79b4e6c56ac6d0a6d8b1e47cf38 [e2e-llm-inference-service] STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e [e2e-llm-inference-service] Installing KServe via kustomize... [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepoolimports.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] Waiting for CRDs to be established... [e2e-llm-inference-service] Waiting for CRD inferenceservices.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD inferenceservices.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD llminferenceserviceconfigs.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD llminferenceserviceconfigs.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD clusterstoragecontainers.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD clusterstoragecontainers.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD datascienceclusters.datasciencecluster.opendatahub.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD datascienceclusters.datasciencecluster.opendatahub.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io condition met [e2e-llm-inference-service] Applying all resources... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepoolimports.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] serviceaccount/kserve-controller-manager serverside-applied [e2e-llm-inference-service] serviceaccount/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-leader-election-role serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/llmisvc-leader-election-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-inferenceservice-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-metrics-reader-cluster-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-proxy-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/llmisvc-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-inferenceservice-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-llmisvc-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/llmisvc-manager-rolebinding serverside-applied [e2e-llm-inference-service] configmap/inferenceservice-config serverside-applied [e2e-llm-inference-service] configmap/kserve-parameters serverside-applied [e2e-llm-inference-service] secret/kserve-webhook-server-secret serverside-applied [e2e-llm-inference-service] secret/mlpipeline-s3-artifact serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-metrics-service serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-service serverside-applied [e2e-llm-inference-service] service/kserve-webhook-server-service serverside-applied [e2e-llm-inference-service] service/llmisvc-controller-manager-service serverside-applied [e2e-llm-inference-service] service/llmisvc-webhook-server-service serverside-applied [e2e-llm-inference-service] service/s3-service serverside-applied [e2e-llm-inference-service] deployment.apps/kserve-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/seaweedfs serverside-applied [e2e-llm-inference-service] networkpolicy.networking.k8s.io/kserve-controller-manager serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] clusterstoragecontainer.serving.kserve.io/default serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-tracing serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceserviceconfig.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io serverside-applied [e2e-llm-inference-service] Waiting for llmisvc-controller-manager to be ready... [e2e-llm-inference-service] Waiting for pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to be created... [e2e-llm-inference-service] Pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" found. [e2e-llm-inference-service] Current pods for -l "control-plane=llmisvc-controller-manager" in namespace "kserve": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] llmisvc-controller-manager-76746f8945-m5v4c 0/1 ContainerCreating 0 7s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to become ready... [e2e-llm-inference-service] pod/llmisvc-controller-manager-76746f8945-m5v4c condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" are ready. [e2e-llm-inference-service] Re-applying LLMInferenceServiceConfig resources with webhook validation... [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-decode-template is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-decode-worker-data-parallel is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-prefill-template is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-prefill-worker-data-parallel is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-scheduler is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-template is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-tracing serverside-applied [e2e-llm-inference-service] Warning: modifying well-known config kserve/kserve-config-llm-worker-data-parallel is not recommended. Consider creating a custom config instead [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] Applying DSC/DSCI resources... [e2e-llm-inference-service] dscinitialization.dscinitialization.opendatahub.io/test-dsci created [e2e-llm-inference-service] datasciencecluster.datasciencecluster.opendatahub.io/test-dsc created [e2e-llm-inference-service] KServe manual installation complete [e2e-llm-inference-service] 🔧 Configuration: [e2e-llm-inference-service] KServe deployment: ❌ disabled [e2e-llm-inference-service] Kuadrant deployment: ✅ enabled [e2e-llm-inference-service] [e2e-llm-inference-service] Checking OpenShift server version...(4.21.23) [e2e-llm-inference-service] 🎯 Server version (4.21.23) is 4.19.9 or higher - continue with the script [e2e-llm-inference-service] ⏳ Installing cert-manager [e2e-llm-inference-service] namespace/cert-manager-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/openshift-cert-manager-operator created [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-cert-manager-operator created [e2e-llm-inference-service] Waiting for openshift-cert-manager-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (5/300) [e2e-llm-inference-service] CSV cert-manager-operator.v1.20.0 found, but not yet Succeeded (Phase: Installing). Waiting... (10/300) [e2e-llm-inference-service] CSV cert-manager-operator.v1.20.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD certificates.cert-manager.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD certificates.cert-manager.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io condition met [e2e-llm-inference-service] ✅ Cert-manager installed [e2e-llm-inference-service] ⏳ Installing openshift-lws-operator [e2e-llm-inference-service] namespace/openshift-lws-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] subscription.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] Waiting for leader-worker-set CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (5/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (10/300) [e2e-llm-inference-service] CSV leader-worker-set.v1.0.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD leaderworkersetoperators.operator.openshift.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD leaderworkersetoperators.operator.openshift.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/leaderworkersetoperators.operator.openshift.io condition met [e2e-llm-inference-service] leaderworkersetoperator.operator.openshift.io/cluster created [e2e-llm-inference-service] ⏳ waiting for openshift-lws-operator to be ready.… [e2e-llm-inference-service] Waiting for pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to be created... [e2e-llm-inference-service] Pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" found. [e2e-llm-inference-service] Current pods for -l "name=openshift-lws-operator" in namespace "openshift-lws-operator": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-lws-operator-fd8ccff4c-d22pr 1/1 Running 0 11s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to become ready... [e2e-llm-inference-service] pod/openshift-lws-operator-fd8ccff4c-d22pr condition met [e2e-llm-inference-service] Pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" are ready. [e2e-llm-inference-service] ✅ openshift-lws-operator installed [e2e-llm-inference-service] gatewayclass.gateway.networking.k8s.io/openshift-default created [e2e-llm-inference-service] Waiting for pod -l "app=istiod" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "app=istiod" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "app=istiod" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] istiod-openshift-gateway-94bb8fbfd-fwgmd 0/1 Running 0 7s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "app=istiod" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/istiod-openshift-gateway-94bb8fbfd-fwgmd condition met [e2e-llm-inference-service] Pod(s) -l "app=istiod" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Creating a Gateway [e2e-llm-inference-service] Error from server (AlreadyExists): namespaces "openshift-ingress" already exists [e2e-llm-inference-service] ⏳ Creating gateway memory ConfigMap for parametersRef (2Gi) [e2e-llm-inference-service] configmap/gateway-proxy-config created [e2e-llm-inference-service] gateway.gateway.networking.k8s.io/openshift-ai-inference created [e2e-llm-inference-service] Waiting for pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-ai-inference-openshift-default-655f7c58b4-4jvzv 0/1 ContainerCreating 0 5s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv condition met [e2e-llm-inference-service] Pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Installing RHCL(Kuadrant) operator [e2e-llm-inference-service] namespace/kuadrant-system created [e2e-llm-inference-service] subscription.operators.coreos.com/rhcl-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/kuadrant created [e2e-llm-inference-service] Waiting for rhcl-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (15/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Pending). Waiting... (20/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (25/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD kuadrants.kuadrant.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD kuadrants.kuadrant.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/kuadrants.kuadrant.io condition met [e2e-llm-inference-service] Waiting for apiserver discovery /apis/kuadrant.io/v1beta1 to list kuadrants (timeout: 120s)… [e2e-llm-inference-service] Discovery for kuadrant.io/v1beta1 includes kuadrants. [e2e-llm-inference-service] ⏳ sleeping 30s after discovery (RESTMapper can trail discovery)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant created [e2e-llm-inference-service] ⏳ waiting for Kuadrant Ready (attempt 1/2, timeout 5m)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant condition met [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-6d85f6564-k7t49 1/1 Running 0 81s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-6d85f6564-k7t49 condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ⏳ waiting for authorino service to be created... [e2e-llm-inference-service] service/authorino-authorino-authorization condition met [e2e-llm-inference-service] service/authorino-authorino-authorization annotated [e2e-llm-inference-service] Warning: resource authorinos/authorino is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] authorino.operator.authorino.kuadrant.io/authorino configured [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-6d85f6564-k7t49 1/1 Running 0 92s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-6d85f6564-k7t49 condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ✅ kuadrant(authorino) installed [e2e-llm-inference-service] Patching ingress domain... [e2e-llm-inference-service] configmap/inferenceservice-config patched [e2e-llm-inference-service] pod "kserve-controller-manager-c8c545f5f-bfmwz" deleted [e2e-llm-inference-service] Waiting for kserve-controller-manager to be ready... [e2e-llm-inference-service] pod/kserve-controller-manager-c8c545f5f-bn6xc condition met [e2e-llm-inference-service] Installing ODH Model Controller manually... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/accounts.nim.opendatahub.io created [e2e-llm-inference-service] serviceaccount/model-serving-api created [e2e-llm-inference-service] serviceaccount/odh-model-controller created [e2e-llm-inference-service] role.rbac.authorization.k8s.io/leader-election-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/account-editor-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/account-viewer-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-prometheus-k8s created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/metrics-reader created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/odh-model-controller-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/proxy-role created [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/odh-model-controller-rolebinding-opendatahub created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/proxy-rolebinding created [e2e-llm-inference-service] configmap/odh-model-controller-parameters created [e2e-llm-inference-service] service/model-serving-api created [e2e-llm-inference-service] service/odh-model-controller-metrics-service created [e2e-llm-inference-service] service/odh-model-controller-webhook-service created [e2e-llm-inference-service] deployment.apps/model-serving-api created [e2e-llm-inference-service] deployment.apps/odh-model-controller created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/model-serving-api-metrics created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/odh-model-controller-metrics-monitor created [e2e-llm-inference-service] template.template.openshift.io/autogluon-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/guardrails-detector-huggingface-serving-template created [e2e-llm-inference-service] template.template.openshift.io/kserve-ovms created [e2e-llm-inference-service] template.template.openshift.io/mlserver-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template-fast-2 created [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/mutating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/validating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] Waiting for deployment "odh-model-controller" rollout to finish: 0 of 1 updated replicas are available... [e2e-llm-inference-service] deployment "odh-model-controller" successfully rolled out [e2e-llm-inference-service] networkpolicy.networking.k8s.io/allow-all created [e2e-llm-inference-service] KServe setup complete (namespace: kserve) [e2e-llm-inference-service] Add testing models to SeaweedFS S3 storage ... [e2e-llm-inference-service] Waiting for SeaweedFS deployment to be ready... [e2e-llm-inference-service] deployment "seaweedfs" successfully rolled out [e2e-llm-inference-service] S3 init job not completed, re-creating... [e2e-llm-inference-service] job.batch/s3-init replaced [e2e-llm-inference-service] Waiting for S3 init job to complete... [e2e-llm-inference-service] job.batch/s3-init condition met [e2e-llm-inference-service] Prepare CI namespace and install ServingRuntimes [e2e-llm-inference-service] Setting up CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Tearing down CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Namespace kserve-ci-e2e-test does not exist, skipping deletion [e2e-llm-inference-service] CI namespace teardown complete [e2e-llm-inference-service] Creating namespace kserve-ci-e2e-test [e2e-llm-inference-service] namespace/kserve-ci-e2e-test created [e2e-llm-inference-service] Applying S3 artifact secret [e2e-llm-inference-service] secret/mlpipeline-s3-artifact created [e2e-llm-inference-service] Applying storage-config secret [e2e-llm-inference-service] secret/storage-config created [e2e-llm-inference-service] Applying SeaweedFS S3 credentials secret [e2e-llm-inference-service] secret/seaweedfs-s3-creds created [e2e-llm-inference-service] Linking seaweedfs-s3-creds to default service account [e2e-llm-inference-service] Creating odh-trusted-ca-bundle configmap [e2e-llm-inference-service] configmap/odh-trusted-ca-bundle created [e2e-llm-inference-service] Installing ServingRuntimes [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-autogluonserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver-multinode created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-lgbserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-mlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-paddleserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-pmmlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-predictiveserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-sklearnserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tensorflow-serving created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-torchserve created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tritonserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-vllmserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-xgbserver created [e2e-llm-inference-service] CI namespace setup complete [e2e-llm-inference-service] Setup complete [e2e-llm-inference-service] === E2E cluster / operator summary === [e2e-llm-inference-service] Client Version: 4.20.11 [e2e-llm-inference-service] Kustomize Version: v5.6.0 [e2e-llm-inference-service] Server Version: 4.21.23 [e2e-llm-inference-service] Kubernetes Version: v1.34.8 [e2e-llm-inference-service] ClusterVersion desired: 4.21.23 [e2e-llm-inference-service] ClusterVersion history (latest): 4.21.23 (Completed) [e2e-llm-inference-service] CSVs in kuadrant-system: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-keda: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] custom-metrics-autoscaler.v2.19.0-1 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in cert-manager-operator: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-lws-operator: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] leader-worker-set.v1.0.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-operators (ODH / shared operators, filtered): [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] Kuadrant / Authorino (diagnostics): [e2e-llm-inference-service] CRD kuadrants.kuadrant.io versions: v1beta1 served=true storage=true [e2e-llm-inference-service] Subscriptions in kuadrant-system: [e2e-llm-inference-service] authorino-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators authorino-operator.v1.4.1 [e2e-llm-inference-service] dns-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators dns-operator.v1.4.0 [e2e-llm-inference-service] limitador-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators limitador-operator.v1.4.0 [e2e-llm-inference-service] rhcl-operator stable redhat-operators rhcl-operator.v1.4.1 [e2e-llm-inference-service] Kuadrant CR conditions (kuadrant/kuadrant-system): [e2e-llm-inference-service] Ready=True (Ready) [e2e-llm-inference-service] KServe deployments in kserve: [e2e-llm-inference-service] kserve-controller-manager: ready=1 image=quay.io/opendatahub/kserve-controller@sha256:f494edae70be745f0175d334523db2d34c4b6619a3e2d1bad8c7a26323eef5b3 [e2e-llm-inference-service] imageID: quay.io/opendatahub/kserve-controller@sha256:e21ea43a31769515c9b364a9bf5a4f2316fa8e1febdfe9273c71294dadc72f94 [e2e-llm-inference-service] odh-model-controller: ready=1 image=quay.io/opendatahub/odh-model-controller:fast [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-model-controller@sha256:5d13ab3c20c8da1064020d19c1d95fd5a563c47cac7c19e4051ee53b992a80da [e2e-llm-inference-service] llmisvc-controller-manager: ready=1 image=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:4c9e95dda047d9c923ada8c9d72c4b97eb988a42011479795898b1bc9b478f4f [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:4c9e95dda047d9c923ada8c9d72c4b97eb988a42011479795898b1bc9b478f4f [e2e-llm-inference-service] === End E2E cluster / operator summary === [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] CA certificate extracted [e2e-llm-inference-service] REQUESTS_CA_BUNDLE=/tmp/ca.crt [e2e-llm-inference-service] Run E2E tests: llminferenceservice and cluster_cpu and not autoscaling and not tracing [e2e-llm-inference-service] Starting E2E functional tests ... [e2e-llm-inference-service] Parallelism requested for pytest is 2 [e2e-llm-inference-service] ============================= test session starts ============================== [e2e-llm-inference-service] platform linux -- Python 3.11.13, pytest-7.4.4, pluggy-1.5.0 -- /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] cachedir: .pytest_cache [e2e-llm-inference-service] metadata: {'Python': '3.11.13', 'Platform': 'Linux-5.14.0-570.122.1.el9_6.x86_64-x86_64-with-glibc2.34', 'Packages': {'pytest': '7.4.4', 'pluggy': '1.5.0'}, 'Plugins': {'metadata': '3.1.1', 'anyio': '4.9.0', 'httpx': '0.30.0', 'xdist': '3.6.1', 'cov': '5.0.0', 'asyncio': '0.23.8', 'json-report': '1.5.0'}, 'PLATFORM': 'el9'} [e2e-llm-inference-service] rootdir: /workspace/source/test/e2e [e2e-llm-inference-service] configfile: pytest.ini [e2e-llm-inference-service] plugins: metadata-3.1.1, anyio-4.9.0, httpx-0.30.0, xdist-3.6.1, cov-5.0.0, asyncio-0.23.8, json-report-1.5.0 [e2e-llm-inference-service] asyncio: mode=Mode.STRICT [e2e-llm-inference-service] created: 2/2 workers [e2e-llm-inference-service] 2 workers [42 items] [e2e-llm-inference-service] [e2e-llm-inference-service] scheduling tests via WorkStealingScheduling [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] 2026-07-08 16:45:49.510 6319 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] 2026-07-08 16:45:49.510 6316 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] 2026-07-08 16:45:49.565 6316 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 16:45:49.565 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 16:45:49.585 6316 kserve.trace Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-08 16:45:49.585 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-08 16:45:49.593 6316 kserve.trace ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-08 16:45:49.593 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] 2026-07-08 16:46:06.890 6316 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 16:46:06.890 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 16:46:06.933 6316 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-08 16:46:06.933 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_enabled_requires_token[cluster_cpu-cluster_single_node-auth-enabled-default] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_enabled_requires_token[cluster_cpu-cluster_single_node-auth-enabled-default] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_invalid_token_rejected[cluster_cpu-cluster_single_node-auth-invalid-token] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_invalid_token_rejected[cluster_cpu-cluster_single_node-auth-invalid-token] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_disabled_no_token_required[cluster_cpu-cluster_single_node-auth-disabled] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_disabled_no_token_required[cluster_cpu-cluster_single_node-auth-disabled] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] 2026-07-08 16:55:59.175 6316 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 16:55:59.175 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 16:55:59.224 6316 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-08 16:55:59.224 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] 2026-07-08 17:04:29.122 6316 kserve.trace Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:04:29.122 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:04:29.156 6316 kserve.trace ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-08 17:04:29.156 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():57] ✓ Successfully updated Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-08 17:04:29.156 6316 kserve.trace Checking HttpRoute router-route-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:04:29.156 6316 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-1 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:04:29.159 6316 kserve.trace Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-08 17:04:29.159 6316 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-08 17:04:29.170 6316 kserve.trace ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-08 17:04:29.170 6316 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-08 17:04:29.170 6316 kserve.trace Checking HttpRoute router-route-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:04:29.170 6316 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:04:29.176 6316 kserve.trace Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-08 17:04:29.176 6316 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-08 17:04:29.185 6316 kserve.trace ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-08 17:04:29.185 6316 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] 2026-07-08 17:17:39.849 6316 kserve.trace Checking Gateway router-gateway-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:17:39.849 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:17:39.875 6316 kserve.trace Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-08 17:17:39.875 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-08 17:17:39.882 6316 kserve.trace ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-08 17:17:39.882 6316 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-08 17:17:39.882 6316 kserve.trace Checking HttpRoute router-route-3 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:17:39.882 6316 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-3 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:17:39.885 6316 kserve.trace Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-08 17:17:39.885 6316 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-08 17:17:39.894 6316 kserve.trace ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-08 17:17:39.894 6316 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-08 17:17:39.894 6316 kserve.trace Checking HttpRoute router-route-4 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:17:39.894 6316 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-4 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] 2026-07-08 17:17:39.897 6316 kserve.trace Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-08 17:17:39.897 6316 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-08 17:17:39.906 6316 kserve.trace ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-08 17:17:39.906 6316 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-single-lora-adapter-hf] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-single-lora-adapter-hf] [e2e-llm-inference-service] llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-multiple-lora-adapters] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-multiple-lora-adapters] [e2e-llm-inference-service] llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_prestop_hook.py::test_prestop_hook[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_prestop_hook.py::test_prestop_hook[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_rolling_upgrade.py::test_rolling_upgrade_coordination[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_rolling_upgrade.py::test_rolling_upgrade_coordination[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw0] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [e2e-llm-inference-service] =================================== FAILURES =================================== [e2e-llm-inference-service] _ test_llm_inference_service[router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] _ [e2e-llm-inference-service] [gw0] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KSer... {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-gateway-ref", [e2e-llm-inference-service] "router-with-managed-route", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="choices"), [e2e-llm-inference-service] expected_gateway=ROUTER_GATEWAYS[0], [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="custom-route-timeout-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="router-with-refs-test", [e2e-llm-inference-service] expected_gateway=ROUTER_GATEWAYS[0], [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[0]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[0], ROUTER_ROUTES[1]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=["router-managed", "workload-pd-cpu", "model-fb-opt-125m"], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-custom-route-timeout-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="custom-route-timeout-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-with-refs-pd", [e2e-llm-inference-service] "scheduler-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="You are an expert in Kubernetes-native machine learning serving platforms, with deep knowledge of the KServe project. " [e2e-llm-inference-service] "Explain the challenges of serving large-scale models, GPU scheduling, and how KServe integrates with capabilities like multi-model serving. " [e2e-llm-inference-service] "Provide a detailed comparison with open source alternatives, focusing on operational trade-offs.", [e2e-llm-inference-service] service_name="router-with-refs-pd-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] expected_gateway=ROUTER_GATEWAYS[1], [e2e-llm-inference-service] before_test=[ [e2e-llm-inference-service] lambda: create_router_resources( [e2e-llm-inference-service] gateways=[ROUTER_GATEWAYS[1]], [e2e-llm-inference-service] routes=[ROUTER_ROUTES[2], ROUTER_ROUTES[3]], [e2e-llm-inference-service] ) [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.custom_gateway, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-dp-ep-gpu", [e2e-llm-inference-service] "workload-dp-ep-prefill-gpu", [e2e-llm-inference-service] "model-deepseek-v2-lite", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="Delve into the multifaceted implications of a fully disaggregated cloud architecture, specifically " [e2e-llm-inference-service] "where the compute plane (P) and the data plane (D) are independently deployed and managed for a " [e2e-llm-inference-service] "geographically distributed, high-throughput, low-latency microservices ecosystem. Beyond the " [e2e-llm-inference-service] "fundamental challenges of network latency and data consistency, elaborate on the advanced " [e2e-llm-inference-service] "considerations and trade-offs inherent in such a setup: 1. Network Architecture and Protocols: " [e2e-llm-inference-service] "How would the network fabric and underlying protocols (e.g., RDMA, custom transport layers) need to " [e2e-llm-inference-service] "evolve to support optimal performance and minimize inter-plane communication overhead, especially for " [e2e-llm-inference-service] "synchronous operations? Discuss the role of network programmability (e.g., SDN, P4) in dynamically " [e2e-llm-inference-service] "optimizing routing and traffic flow between P and D. 2. Advanced Data Consistency and Durability: " [e2e-llm-inference-service] "Explore sophisticated data consistency models (e.g., causal consistency, strong eventual consistency) " [e2e-llm-inference-service] "and their applicability in balancing performance and data integrity across a globally distributed data plane. " [e2e-llm-inference-service] "Detail strategies for ensuring data durability and fault tolerance, including multi-region replication, " [e2e-llm-inference-service] "intelligent partitioning, and recovery mechanisms in the event of partial or full plane failures. " [e2e-llm-inference-service] "3. Dynamic Resource Orchestration and Cost Optimization: Analyze how an orchestration layer would intelligently " [e2e-llm-inference-service] "manage the independent scaling of compute (P) and data (D) resources, considering fluctuating workloads, " [e2e-llm-inference-service] "cost efficiency, and performance targets (e.g., using predictive analytics for resource provisioning). " [e2e-llm-inference-service] "Discuss mechanisms for dynamically reallocating compute nodes to different data partitions based on " [e2e-llm-inference-service] "workload patterns and data locality, potentially involving live migration strategies. " [e2e-llm-inference-service] "4. Security and Compliance in a Distributed Landscape: Address the enhanced security perimeter " [e2e-llm-inference-service] "challenges, including securing communication channels between P and D (encryption in transit, mutual TLS), " [e2e-llm-inference-service] "fine-grained access control to data at rest and in motion, and identity management across disaggregated " [e2e-llm-inference-service] "components. Discuss how such an architecture impacts compliance with regulatory frameworks (e.g., GDPR, HIPAA) " [e2e-llm-inference-service] "concerning data sovereignty, privacy, and auditability. 5. Operational Complexity and Observability: " [e2e-llm-inference-service] "Examine the increased complexity in monitoring, logging, and tracing across highly decoupled compute and " [e2e-llm-inference-service] "data planes. What specialized tooling and practices (e.g., distributed tracing with OpenTelemetry, advanced AIOps) " [e2e-llm-inference-service] "would be essential? How would incident response and troubleshooting differ in this disaggregated environment " [e2e-llm-inference-service] "compared to traditional integrated systems? Consider the challenges of pinpointing root causes across " [e2e-llm-inference-service] "independent failures. 6. Real-world Applicability and Future Trends: Identify specific industries " [e2e-llm-inference-service] "or use cases (e.g., high-frequency trading, IoT edge processing, large language model inference) " [e2e-llm-inference-service] "where the benefits of P/D disaggregation would strongly outweigh its complexities. " [e2e-llm-inference-service] "Conclude by speculating on emerging technologies or paradigms (e.g., serverless compute functions " [e2e-llm-inference-service] "directly interacting with object storage, in-memory disaggregation) that could further drive or " [e2e-llm-inference-service] "transform P/D disaggregation in cloud computing.", [e2e-llm-inference-service] max_tokens=2000, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_gpu, [e2e-llm-inference-service] pytest.mark.cluster_nvidia, [e2e-llm-inference-service] pytest.mark.cluster_nvidia_roce, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-no-scheduler", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.no_scheduler, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-simulated-dp-ep-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="This test simulates DP+EP that can run on CPU, the idea is to test the LWS-based deployment, " [e2e-llm-inference-service] "but without the resources requirements for DP+EP (GPUs and ROCe/IB).", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_multi_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Scheduler config tests [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-inline-config-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Chat completions endpoint coverage [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] "model-qwen2.5-0.5b", [e2e-llm-inference-service] ], [e2e-llm-inference-service] model_name="Qwen/Qwen2.5-0.5B-Instruct", [e2e-llm-inference-service] endpoint="/v1/chat/completions", [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] payload_formatter=chat_completions_payload, [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="choices"), [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-configmap-ref", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-configmap-ref-test", [e2e-llm-inference-service] before_test=[create_scheduler_configmap], [e2e-llm-inference-service] after_test=[delete_scheduler_configmap], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-replicas", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-ha-replicas-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-custom-template", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-custom-template-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[pytest.mark.cluster_cpu, pytest.mark.cluster_single_node], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Scheduler v0.6 → v0.7 migration tests. [e2e-llm-inference-service] # Deploy v0.6-style configs and verify the controller migrates them [e2e-llm-inference-service] # so the v0.7 scheduler boots successfully. [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-v06-pd-config-migration", [e2e-llm-inference-service] "workload-llmd-simulator-pd", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-v06-pd-migration-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-v06-nonzero-threshold-migration", [e2e-llm-inference-service] "workload-llmd-simulator-pd", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="scheduler-v06-threshold-migration-test", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Precise prefix KV cache routing test [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "scheduler-with-precise-prefix-cache-inline-config", [e2e-llm-inference-service] "workload-llmd-simulator-kvcache", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="precise-prefix-cache-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Models endpoint coverage [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/models", [e2e-llm-inference-service] response_assertion=create_response_assertion(with_field="data"), [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — /v1/completions [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches("facebook/opt-125m"), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] }, [e2e-llm-inference-service] peers=[ [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] "model-qwen2.5-0.5b", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches( [e2e-llm-inference-service] "Qwen/Qwen2.5-0.5B-Instruct" [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/Qwen/Qwen2.5-0.5B-Instruct", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — /v1/chat/completions [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/chat/completions", [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] payload_formatter=chat_completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches("facebook/opt-125m"), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] }, [e2e-llm-inference-service] peers=[ [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator", [e2e-llm-inference-service] "model-qwen2.5-0.5b", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/chat/completions", [e2e-llm-inference-service] prompt="What is KServe?", [e2e-llm-inference-service] payload_formatter=chat_completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches( [e2e-llm-inference-service] "Qwen/Qwen2.5-0.5B-Instruct" [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/Qwen/Qwen2.5-0.5B-Instruct", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — LoRA adapter [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m-with-lora-hf", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/completions", [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] model_name=f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1", [e2e-llm-inference-service] payload_formatter=completions_payload, [e2e-llm-inference-service] response_assertion=assert_model_field_matches( [e2e-llm-inference-service] f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1" [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] pytest.mark.lora, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # Model-based routing via X-Gateway-Model-Name header — /v1/models (base + LoRA) [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m-with-lora-hf", [e2e-llm-inference-service] ], [e2e-llm-inference-service] endpoint="/v1/models", [e2e-llm-inference-service] response_assertion=assert_models_contains( [e2e-llm-inference-service] "facebook/opt-125m", [e2e-llm-inference-service] f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] "lora-adapter-1", [e2e-llm-inference-service] f"publishers/{KSERVE_TEST_NAMESPACE}/models/lora-adapter-1", [e2e-llm-inference-service] ), [e2e-llm-inference-service] url_getter=get_model_routing_url, [e2e-llm-inference-service] extra_headers={ [e2e-llm-inference-service] MODEL_ROUTING_HEADER: f"publishers/{KSERVE_TEST_NAMESPACE}/models/facebook/opt-125m", [e2e-llm-inference-service] }, [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.model_routing, [e2e-llm-inference-service] pytest.mark.lora, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] # PVC storage tests -- validate direct PVC volume mount with real vLLM serving [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-pvc", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] before_test=[ensure_pvc_with_model], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.pvc_storage, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-pd-cpu", [e2e-llm-inference-service] "model-pvc", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] response_assertion=assert_200_with_choices, [e2e-llm-inference-service] before_test=[ensure_pvc_with_model], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.pvc_storage, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-simulated-dp-ep-cpu", [e2e-llm-inference-service] "model-pvc", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] before_test=[ensure_pvc_with_model], [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_multi_node, [e2e-llm-inference-service] pytest.mark.pvc_storage, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_inference_service(test_case: TestCase): # noqa: F811 [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = KServeClient( [e2e-llm-inference-service] config_file=os.environ.get("KUBECONFIG", "~/.kube/config"), [e2e-llm-inference-service] client_configuration=client.Configuration(), [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] if not test_case.llm_service.metadata.annotations: [e2e-llm-inference-service] test_case.llm_service.metadata.annotations = {} [e2e-llm-inference-service] [e2e-llm-inference-service] test_case.llm_service.metadata.annotations[ [e2e-llm-inference-service] "security.opendatahub.io/enable-auth" [e2e-llm-inference-service] ] = "false" [e2e-llm-inference-service] prefix = test_case.log_prefix [e2e-llm-inference-service] [e2e-llm-inference-service] test_failed = False [e2e-llm-inference-service] try: [e2e-llm-inference-service] print(f"{prefix} Creating LLMInferenceService {service_name}") [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] print(f"{prefix} Waiting for LLMInferenceService {service_name} to be ready") [e2e-llm-inference-service] wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] print(f"{prefix} Waiting for model response from {service_name}") [e2e-llm-inference-service] > wait_for_model_response( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] test_case, [e2e-llm-inference-service] test_case.wait_timeout, [e2e-llm-inference-service] extra_headers=test_case.extra_headers, [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:816: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, TestCase(base_refs=['router-managed', 'scheduler-v0... {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m'), 900) [e2e-llm-inference-service] kwargs = {'extra_headers': None}, func_name = 'wait_for_model_response' [e2e-llm-inference-service] timestamp_start = '2026-07-08T17:35:24.461383', start_time = 1783532124.4616473 [e2e-llm-inference-service] duration = 1102.97026014328, timestamp_end = '2026-07-08T17:53:47.431910' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KSer... {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] timeout_seconds = 900, extra_headers = None [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_model_response( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] test_case: TestCase, # noqa: F811 [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] extra_headers: Optional[Dict[str, str]] = None, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def get_successful_response(): [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_case.url_getter: [e2e-llm-inference-service] service_url = test_case.url_getter(kserve_client, test_case.llm_service) [e2e-llm-inference-service] else: [e2e-llm-inference-service] service_url = get_llm_service_url(kserve_client, test_case.llm_service) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to get service URL: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] model_url = service_url + test_case.endpoint [e2e-llm-inference-service] [e2e-llm-inference-service] headers = {"Content-Type": "application/json"} [e2e-llm-inference-service] if extra_headers: [e2e-llm-inference-service] headers.update(extra_headers) [e2e-llm-inference-service] [e2e-llm-inference-service] if test_case.payload_formatter is not None: [e2e-llm-inference-service] test_payload = test_case.payload_formatter(test_case) [e2e-llm-inference-service] elif test_case.prompt is not None: [e2e-llm-inference-service] test_payload = { [e2e-llm-inference-service] "model": test_case.model_name [e2e-llm-inference-service] if not extra_headers or MODEL_ROUTING_HEADER not in extra_headers [e2e-llm-inference-service] else extra_headers[MODEL_ROUTING_HEADER], [e2e-llm-inference-service] "prompt": test_case.prompt, [e2e-llm-inference-service] "max_tokens": test_case.max_tokens, [e2e-llm-inference-service] } [e2e-llm-inference-service] else: [e2e-llm-inference-service] test_payload = None [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Calling LLM service at {model_url} with payload {test_payload}") [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_payload is not None: [e2e-llm-inference-service] response = post_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] json_data=test_payload, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] else: [e2e-llm-inference-service] response = get_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] logger.error(f"❌ Failed to call model: {e}") [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to call model: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Model response is {response.status_code}: {response.text[:500]}") [e2e-llm-inference-service] [e2e-llm-inference-service] if 200 <= response.status_code < 300: [e2e-llm-inference-service] return response [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Service returned {response.status_code}: {response.text}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] > response = wait_for(get_successful_response, timeout=timeout_seconds, interval=5.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1119: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .get_successful_response at 0x7f0ea1295440> [e2e-llm-inference-service] timeout = 900, interval = 5.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] last_msg = None [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1215: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def get_successful_response(): [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_case.url_getter: [e2e-llm-inference-service] service_url = test_case.url_getter(kserve_client, test_case.llm_service) [e2e-llm-inference-service] else: [e2e-llm-inference-service] service_url = get_llm_service_url(kserve_client, test_case.llm_service) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to get service URL: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] model_url = service_url + test_case.endpoint [e2e-llm-inference-service] [e2e-llm-inference-service] headers = {"Content-Type": "application/json"} [e2e-llm-inference-service] if extra_headers: [e2e-llm-inference-service] headers.update(extra_headers) [e2e-llm-inference-service] [e2e-llm-inference-service] if test_case.payload_formatter is not None: [e2e-llm-inference-service] test_payload = test_case.payload_formatter(test_case) [e2e-llm-inference-service] elif test_case.prompt is not None: [e2e-llm-inference-service] test_payload = { [e2e-llm-inference-service] "model": test_case.model_name [e2e-llm-inference-service] if not extra_headers or MODEL_ROUTING_HEADER not in extra_headers [e2e-llm-inference-service] else extra_headers[MODEL_ROUTING_HEADER], [e2e-llm-inference-service] "prompt": test_case.prompt, [e2e-llm-inference-service] "max_tokens": test_case.max_tokens, [e2e-llm-inference-service] } [e2e-llm-inference-service] else: [e2e-llm-inference-service] test_payload = None [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Calling LLM service at {model_url} with payload {test_payload}") [e2e-llm-inference-service] try: [e2e-llm-inference-service] if test_payload is not None: [e2e-llm-inference-service] response = post_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] json_data=test_payload, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] else: [e2e-llm-inference-service] response = get_with_retry( [e2e-llm-inference-service] model_url, [e2e-llm-inference-service] headers=headers, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] except Exception as e: [e2e-llm-inference-service] logger.error(f"❌ Failed to call model: {e}") [e2e-llm-inference-service] raise AssertionError(f"❌ Failed to call model: {e}") from e [e2e-llm-inference-service] [e2e-llm-inference-service] logger.info(f"Model response is {response.status_code}: {response.text[:500]}") [e2e-llm-inference-service] [e2e-llm-inference-service] if 200 <= response.status_code < 300: [e2e-llm-inference-service] return response [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Service returned {response.status_code}: {response.text}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Service returned 502: [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1115: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1637 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1586 Checking LLMInferenceServiceConfig router-managed-scheduler-v06-pd-5a5debb2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1612 Resource not found, creating LLMInferenceServiceConfig router-managed-scheduler-v06-pd-5a5debb2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1622 ✓ Successfully created LLMInferenceServiceConfig router-managed-scheduler-v06-pd-5a5debb2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1586 Checking LLMInferenceServiceConfig scheduler-v06-pd-config-migrati-bb3264a2 in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1612 Resource not found, creating LLMInferenceServiceConfig scheduler-v06-pd-config-migrati-bb3264a2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1622 ✓ Successfully created LLMInferenceServiceConfig scheduler-v06-pd-config-migrati-bb3264a2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1586 Checking LLMInferenceServiceConfig workload-llmd-simulator-pd-sche-64d0fc8b in namespace kserve-ci-e2e-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1612 Resource not found, creating LLMInferenceServiceConfig workload-llmd-simulator-pd-sche-64d0fc8b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1622 ✓ Successfully created LLMInferenceServiceConfig workload-llmd-simulator-pd-sche-64d0fc8b [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_inference_service] [2026-07-08T17:33:53.514497] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KServe is a', service_name='scheduler-v06-pd-migration-test', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1637 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-07-08T17:33:53.527346] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-07-08T17:33:53.578690] end - ✅ in 0.051s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-07-08T17:33:53.578828] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: Missing true conditions: {'RouterReady', 'WorkloadsReady', 'Ready'}, expected {'RouterReady', 'WorkloadsReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-08T17:34:50Z', 'message': 'The following HTTPRoutes are not ready: [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'severity': 'Info', 'status': 'False', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'message': 'Inference Pool kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool exists but no Gateway controller has accepted it yet', 'reason': 'WaitingForGateway', 'severity': 'Info', 'status': 'False', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'message': 'The following HTTPRoutes are not ready: [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'message': 'The following HTTPRoutes are not ready: [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'reason': 'Progressing', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: Missing true conditions: {'RouterReady', 'Ready'}, expected {'RouterReady', 'WorkloadsReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-08T17:35:01Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-07-08T17:35:01Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-07-08T17:35:01Z', 'severity': 'Info', 'status': 'True', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-08T17:35:01Z', 'severity': 'Info', 'status': 'True', 'type': 'PrefillWorkloadReady'}, {'lastTransitionTime': '2026-07-08T17:34:50Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-08T17:35:01Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-08T17:35:01Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-08T17:35:01Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-07-08T17:35:01Z', 'status': 'True', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [wait_for_llm_isvc_ready] [2026-07-08T17:35:24.461241] end - ✅ in 90.882s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_model_response] [2026-07-08T17:35:24.461383] start - args=(, TestCase(base_refs=['router-managed', 'scheduler-v06-pd-config-migration', 'workload-llmd-simulator-pd'], prompt='KServe is a', service_name='scheduler-v06-pd-migration-test', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m'), 900), kwargs={'extra_headers': None} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llm_service_url] [2026-07-08T17:35:24.461656] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llm_service_url] [2026-07-08T17:35:24.468342] end - ✅ in 0.006s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1092 Calling LLM service at http://a5270213f0a684f169b1b83d9db8a952-2022145956.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions with payload {'model': 'facebook/opt-125m', 'prompt': 'KServe is a', 'max_tokens': 20} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1111 Model response is 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1222 Waiting: Service returned 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llm_service_url] [2026-07-08T17:41:33.787831] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llm_service_url] [2026-07-08T17:41:33.836012] end - ✅ in 0.048s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1092 Calling LLM service at http://a5270213f0a684f169b1b83d9db8a952-2022145956.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions with payload {'model': 'facebook/opt-125m', 'prompt': 'KServe is a', 'max_tokens': 20} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1111 Model response is 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llm_service_url] [2026-07-08T17:47:43.113481] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'false'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'scheduler-v06-pd-migration-test', [e2e-llm-inference-service] 'namespace': 'kserve-ci-e2e-test', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-scheduler-v06-pd-5a5debb2'}, [e2e-llm-inference-service] {'name': 'scheduler-v06-pd-config-migrati-bb3264a2'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-pd-sche-64d0fc8b'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llm_service_url] [2026-07-08T17:47:43.164039] end - ✅ in 0.050s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1092 Calling LLM service at http://a5270213f0a684f169b1b83d9db8a952-2022145956.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions with payload {'model': 'facebook/opt-125m', 'prompt': 'KServe is a', 'max_tokens': 20} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1111 Model response is 502: [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:1219 Timed out waiting: Service returned 502: [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_model_response] [2026-07-08T17:53:47.431910] end - ❌ 1102.970s: Service returned 502: [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:831 [router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] ❌ ERROR: Failed to call llm inference service scheduler-v06-pd-migration-test: Service returned 502: [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1240 🔍 # Diagnostics for 'scheduler-v06-pd-migration-test' in 'kserve-ci-e2e-test' [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1241 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1242 # LLMInferenceService scheduler-v06-pd-migration-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1245 apiVersion: serving.kserve.io/v1alpha1 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] security.opendatahub.io/enable-auth: 'false' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:33:53Z' [e2e-llm-inference-service] finalizers: [e2e-llm-inference-service] - serving.kserve.io/llmisvc-finalizer [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:security.opendatahub.io/enable-auth: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:baseRefs: {} [e2e-llm-inference-service] manager: OpenAPI-Generator [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:33:53Z' [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:finalizers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] v:"serving.kserve.io/llmisvc-finalizer": {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:33:53Z' [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:addresses: {} [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-decode-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-decode-worker-data-parallel: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-prefill-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-prefill-worker-data-parallel: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-router-route: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-scheduler: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-tracing: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-worker-data-parallel: {} [e2e-llm-inference-service] f:appliedConfigs: {} [e2e-llm-inference-service] f:conditions: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:router: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:gateways: {} [e2e-llm-inference-service] f:scheduler: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:inferencePool: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:service: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:url: {} [e2e-llm-inference-service] f:workloads: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:prefill: {} [e2e-llm-inference-service] f:primary: {} [e2e-llm-inference-service] f:scheduler: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-08T17:35:23Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] resourceVersion: '66502' [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] spec: [e2e-llm-inference-service] baseRefs: [e2e-llm-inference-service] - name: router-managed-scheduler-v06-pd-5a5debb2 [e2e-llm-inference-service] - name: scheduler-v06-pd-config-migrati-bb3264a2 [e2e-llm-inference-service] - name: workload-llmd-simulator-pd-sche-64d0fc8b [e2e-llm-inference-service] model: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uri: '' [e2e-llm-inference-service] status: [e2e-llm-inference-service] addresses: [e2e-llm-inference-service] - name: gateway-external-model-routing [e2e-llm-inference-service] url: http://a5270213f0a684f169b1b83d9db8a952-2022145956.us-east-1.elb.amazonaws.com/ [e2e-llm-inference-service] - name: gateway-external [e2e-llm-inference-service] url: http://a5270213f0a684f169b1b83d9db8a952-2022145956.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] - name: gateway-internal-model-routing [e2e-llm-inference-service] url: http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/ [e2e-llm-inference-service] - name: gateway-internal [e2e-llm-inference-service] url: http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/config-llm-decode-template: kserve-config-llm-decode-template [e2e-llm-inference-service] serving.kserve.io/config-llm-decode-worker-data-parallel: kserve-config-llm-decode-worker-data-parallel [e2e-llm-inference-service] serving.kserve.io/config-llm-prefill-template: kserve-config-llm-prefill-template [e2e-llm-inference-service] serving.kserve.io/config-llm-prefill-worker-data-parallel: kserve-config-llm-prefill-worker-data-parallel [e2e-llm-inference-service] serving.kserve.io/config-llm-router-route: kserve-config-llm-router-route [e2e-llm-inference-service] serving.kserve.io/config-llm-scheduler: kserve-config-llm-scheduler [e2e-llm-inference-service] serving.kserve.io/config-llm-template: kserve-config-llm-template [e2e-llm-inference-service] serving.kserve.io/config-llm-tracing: kserve-config-llm-tracing [e2e-llm-inference-service] serving.kserve.io/config-llm-worker-data-parallel: kserve-config-llm-worker-data-parallel [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: HTTPRoutesReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: InferencePoolReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: MainWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: PrefillWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: PresetsCombined [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:23Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Ready [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:23Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: RouterReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:23Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: SchedulerWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: WorkloadsReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] url: http://a5270213f0a684f169b1b83d9db8a952-2022145956.us-east-1.elb.amazonaws.com/kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:44 TIME NAMESPACE SOURCE TYPE REASON MESSAGE [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:45 -------------------------------------------------------------------------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-6f5f7c596-r2648 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.51/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:28 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:28 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:28 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.51:8000/health": dial tcp 10.134.0.51:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:39 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-disabled-test-kserve-6f5f7c596-r2648 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler-587d7dfcd5-xqq7p to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.38/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:25 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:25 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:25 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-disabled-test-kserve-router-scheduler-587d7dfcd5-xqq7p [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-disabled-test-kserve-router-scheduler-587d7dfcd5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-disabled-test-kserve-6f5f7c596 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:19 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy auth-disabled-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "auth-disabled-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-disabled-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/auth-disabled-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/auth-disabled-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/auth-disabled-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-disabled-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-disabled-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-disabled-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/auth-disabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/auth-disabled-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:33 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [auth-disabled-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-disabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-75d9ddf5db-r282k to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.46/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:18 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulling Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 45.178s (45.178s including waiting). Image size: 3531177328 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:54 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.46:8000/health": dial tcp 10.134.0.46:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:09 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-enabled-test-kserve-75d9ddf5db-r282k [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler-685d575668-8jcm7 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:15 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.34/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:15 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:15 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:15 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:16 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:16 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:16 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:09 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:09 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-enabled-test-kserve-router-scheduler-685d575668-8jcm7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-enabled-test-kserve-router-scheduler-685d575668 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-enabled-test-kserve-75d9ddf5db from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/auth-enabled-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/auth-enabled-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/auth-enabled-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-enabled-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-enabled-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:24 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-enabled-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/auth-enabled-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:05 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [auth-enabled-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:09 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-enabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-84bbd6975d-6mqgf to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.49/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.49:8000/health": dial tcp 10.134.0.49:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:09 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.49:8000/health": dial tcp 10.134.0.49:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-invalid-token-test-kserve-84bbd6975d-6mqgf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler-f95954846-j57pn to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.37/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: auth-invalid-token-test-kserve-router-scheduler-f95954846-j57pn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-invalid-token-test-kserve-router-scheduler-f95954846 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-invalid-token-test-kserve-84bbd6975d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/auth-invalid-token-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/auth-invalid-token-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/auth-invalid-token-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/auth-invalid-token-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/auth-invalid-token-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:08 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/auth-invalid-token-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/auth-invalid-token-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/auth-invalid-token-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:52 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [auth-invalid-token-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-auth-invalid-token-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:52 kserve-ci-e2e-test LLMInferenceServiceController Warning UpdateFailed Failed to update status for LLMInferenceService "conv-test-criticality": llminferenceservices.serving.kserve.io "conv-test-criticality" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:52 kserve-ci-e2e-test LLMInferenceServiceController Warning UpdateFailed Failed to update status for LLMInferenceService "conv-test-lora-crit": llminferenceservices.serving.kserve.io "conv-test-lora-crit" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-5b8f759b49-b5l99 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.65/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:16:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.65:8001/health": dial tcp 10.134.0.65:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-pd-test-kserve-5b8f759b49-b5l99 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-847796cfb8-n9ls5 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.66/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:16:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.66:8000/health": dial tcp 10.134.0.66:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-pd-test-kserve-prefill-847796cfb8-n9ls5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-847796cfb8 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-bfb9dctn6l [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-bfb9dctn6l to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.51/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:43 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:43 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:43 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:19 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:19 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-bfb9dbc8b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-pd-test-kserve-5b8f759b49 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:38 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:58 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:03 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [custom-route-timeout-pd-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-745c57659b-6k7dg to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.56/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:54 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.56:8000/health": dial tcp 10.134.0.56:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:08 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:24 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.56:8000/health": dial tcp 10.134.0.56:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-test-kserve-745c57659b-6k7dg [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: custom-route-timeout-test-kserve-router-scheduler-85cd886dfv82c [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-85cd886dfv82c to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.45/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:55 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:55 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:55 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:08 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:08 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-85cd886d4c from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set custom-route-timeout-test-kserve-745c57659b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:52 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:54 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:10 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:04 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [custom-route-timeout-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:08 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/e2e-pvc-model-download-lnwcc to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:51 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:00 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.44/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:46 kserve-ci-e2e-test job-controller Normal SuccessfulCreate Created pod: e2e-pvc-model-download-lnwcc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:09 kserve-ci-e2e-test job-controller Normal Completed Job completed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:46 kserve-ci-e2e-test persistentvolume-controller Normal WaitForFirstConsumer waiting for first consumer to be created before binding [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:46 kserve-ci-e2e-test persistentvolume-controller Normal ExternalProvisioning Waiting for a volume to be created either by the external provisioner 'ebs.csi.aws.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:46 kserve-ci-e2e-test ebs.csi.aws.com_aws-ebs-csi-driver-controller-5b6f4d8d9f-r89fq_f38c6655-27b4-44fd-995b-f69a70721273 Normal Provisioning External provisioner is provisioning volume for claim "kserve-ci-e2e-test/e2e-pvc-model-storage" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:48 kserve-ci-e2e-test ebs.csi.aws.com_aws-ebs-csi-driver-controller-5b6f4d8d9f-r89fq_f38c6655-27b4-44fd-995b-f69a70721273 Normal ProvisioningSucceeded Successfully provisioned volume pvc-42130d34-0182-4d20-95af-c345cf3e144c [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:04 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc56vlhc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc56vlhc to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:04 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.31/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:04 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.178s (1.178s including waiting). Image size: 98346788 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:36 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:04 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-7c9c8cffc5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b8zlzv [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b8zlzv to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.45/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:40 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.204s (1.205s including waiting). Image size: 98346788 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:41 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:41 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-7bc8dd6c5b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:41 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-3c960099-kserve-69d7c86dbcwbcz6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-69d7c86dbcwbcz6 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.71/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:02 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.71:8000/health": dial tcp 10.134.0.71:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:35 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.71:8000/health": dial tcp 10.134.0.71:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:41 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-3c960099-kserve-69d7c86dbc from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:39 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-3c960099-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-3c960099-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-3c960099-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:22:09 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvd10b93c8eba12f87c3a350a5cae2ee0c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:12 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-3c960099] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-85f84c97d6gjlxz [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-85f84c97d6gjlxz to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.62/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:12:41 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.62:8001/health": dial tcp 10.134.0.62:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:07 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:07 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6chnh54 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6chnh54 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.63/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:59 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:59 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:59 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:12:41 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.63:8000/health": dial tcp 10.134.0.63:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:07 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-6c8fc44f49 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-50bc673d-kserve-85f84c97d6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:49 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:50 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:16 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:16 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:12:51 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-50bc673d] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test00d7278d8a22c4e39146a6b0eb840f45-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5f4b64bcd658msw [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5f4b64bcd658msw to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.54/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:11 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:11 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:11 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:05 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.54:8000/health": dial tcp 10.134.0.54:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:21 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:36 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.54:8000/health": dial tcp 10.134.0.54:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5f4b64bcd6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:16 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-87882a8e] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test21fe6730fe484f3a92b1a16afe1bac8f-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.55/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:49 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulling Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:38 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 48.688s (48.688s including waiting). Image size: 3531177328 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:38 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:38 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:40 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.73/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:26:24 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.73:8000/health": dial tcp 10.134.0.73:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:24 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test leaderworkerset Normal CreatingRevision Creating revision with key 5f79d656cb for a newly created LeaderWorkerSet [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created leader statefulset llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Replicas are progressing, with 0 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:26:34 kserve-ci-e2e-test leaderworkerset Normal AllGroupsReady All replicas are ready, with 1 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:42 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-scc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:19 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-dc21cb14] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:24 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test386bd5808c5c450f11fd8632e1f821fb-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:07 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4p8kfj [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4p8kfj to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:08 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.31/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:08 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:08 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:08 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:58:40 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:58:55 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.31:8000/health": dial tcp 10.133.0.31:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:07 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-66874c76d4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:04 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:08 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:08 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:18 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-route-e95b1dc1] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-5b7674d95-n5k4k to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.52/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:11 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:11 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:11 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:24 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.52:8000/health": dial tcp 10.134.0.52:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-5b7674d95-n5k4k [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schedm84p to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:05 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.40/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schedm84p [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-75c9b756c4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-5b7674d95 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:02 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:20 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:20 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:35 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-with-7ca60146] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-597cd449d8-tlqgv to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.53/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:41 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:41 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:41 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:58:54 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.53:8000/health": dial tcp 10.134.0.53:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:30 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:46 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.53:8000/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-597cd449d8-tlqgv [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheqqzrd [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-6cc7b8c595 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheqqzrd to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.42/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:35 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:35 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:35 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:36 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:36 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:36 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:30 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:30 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-597cd449d8 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:31 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:57:41 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:09 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-fb-opt-125m-with-ba4d693a] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.61/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:35 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:07 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.60/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:12 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:32 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.60:8000/health": dial tcp 10.134.0.60:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:18 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:34 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.60:8000/health": dial tcp 10.134.0.60:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test leaderworkerset Normal CreatingRevision Creating revision with key 686d468674 for a newly created LeaderWorkerSet [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created leader statefulset llmisvc-model-pvc-router-manage-2577e794-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Replicas are progressing, with 0 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test statefulset-controller Normal SuccessfulCreate create Pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 in StatefulSet llmisvc-model-pvc-router-manage-2577e794-kserve-mn successful [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test leaderworkerset Normal GroupsProgressing Created worker statefulset for leader pod llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:42 kserve-ci-e2e-test leaderworkerset Normal AllGroupsReady All replicas are ready, with 1 groups ready of total 1 groups [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:04 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-2577e794-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-2577e794-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-scc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-mn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:44 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-2577e794-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:55 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-pvc-router-manage-2577e794] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test37591b20e96e9663d45a730d03070f1e-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d6f29q [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d6f29q to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:22 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:25 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.55/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:25 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:25 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:25 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:15 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.55:8000/health": dial tcp 10.134.0.55:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:29 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:45 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.55:8000/health": dial tcp 10.134.0.55:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-59b9d263-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-59b9d263-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv8bf079eb6eda4debfb4ef5bb7817824c-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:40 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-59b9d263-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:25 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-pvc-router-manage-59b9d263] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testc9569cf4801efc0ed27b2f25ffaee875-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-j29rn to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:14 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:17 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.57/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:18 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 884ms (884ms including waiting). Image size: 44914394 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:18 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:18 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:05:57 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.57:8001/health": dial tcp 10.134.0.57:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:22 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:22 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-j29rn [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79lcxhc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79lcxhc to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:14 kserve-ci-e2e-test attachdetach-controller Normal SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:17 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.58/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:05:57 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.58:8000/health": dial tcp 10.134.0.58:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:22 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79497db4cc from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:10 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-pvc-router-manage-e8706282-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-pvc-router-manage-e8706282-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvb19f98874e050eec8ca94d49676113f0-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:32 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-pvc-router-manage-e8706282-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:08 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-pvc-router-manage-e8706282] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:22 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testf5d060a5eb39a04e074b78907a1556a6-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4wjjs9 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4wjjs9 to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.37/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:46 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-797dfc7ff4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:09 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:45 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4c1ba212] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:46 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test80f99418f0f3eb4c80f02e6144a6d4f2-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fh6pz6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98fh6pz6 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.39/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:11 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5c54ddb98f from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:51 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:51 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:51 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:11 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-4f8c0978] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test76d7f532acb694e4a7bcef75d32cd8a1-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd8hk6m [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd8hk6m to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.35/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:11 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-5d8ffd58dd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:15 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:34 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-qwen2-5-0-5b-rout-a50492e9-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:11 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-model-qwen2-5-0-5b-rout-a50492e9] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testb7025ca4d8a6f8f5b2fd08b5581d2678-llmisvc-mode [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-m4nls to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.29/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:31 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:00 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56-m4nls [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:00 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-router-managed-test-llm-4b931143-kserve-bd545d56 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:57 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-4b931143-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-4b931143-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:34 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-router-managed-test-llm-4b931143] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4lthqm [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4lthqm to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.28/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:31 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-router-managed-test-llm-5b1e8f15-kserve-64df7bddb4 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:27 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-5b1e8f15-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-5b1e8f15-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:08 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-router-managed-test-llm-5b1e8f15] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:31 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cfhhm [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879cfhhm to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:20 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.27/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:21 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.033s (1.033s including waiting). Image size: 98346788 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:21 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:21 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:56 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc-router-managed-test-llm-e45d1f79-kserve-7fdbbd4879 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:16 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-e45d1f79-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-e45d1f79-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:30 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:55 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [llmisvc-router-managed-test-llm-e45d1f79] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schert2rf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:37 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-scheduler-6fcb489785 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schert2rf to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:38 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.30/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:38 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:39 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.554s (1.554s including waiting). Image size: 75073927 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:39 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:53:39 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:11 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:11 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schefzx4x [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:05 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-scheduler-7dbcb75dbc from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc1dc4269d1ada5f2d28562215d180c57f-kserve-router-schefzx4x to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:06 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.34/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:06 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:06 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:06 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:07 kserve-ci-e2e-test attachdetach-controller Warning FailedAttachVolume Multi-Attach error for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0, llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0-1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:18 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:10:18 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schexrfjx [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-scheduler-8645f969f6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-router-schexrfjx to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.56/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:44 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:24 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:24 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche5nx8d to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.38/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:14 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:14 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:46 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:46 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-sche5nx8d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:13 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-scheduler-79866dc455 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche597md to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.49/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:52 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:52 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:52 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc44d181485fad85e662eb092f3749502f-kserve-router-sche597md [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:51 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc44d181485fad85e662eb092f3749502f-kserve-router-scheduler-6776999b96 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche852qw to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:20 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.47/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:20 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 2.861s (2.861s including waiting). Image size: 75073927 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:56 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:56 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-sche852qw [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5dd88bfbb7 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schegkwvg [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-scheduler-5f555d4d85 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvc8bf079eb6eda4debfb4ef5bb7817824c-kserve-router-schegkwvg to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:20 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.32/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:00:22 kserve-ci-e2e-test attachdetach-controller Warning FailedAttachVolume Multi-Attach error for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-59b9d263-kserve-585587bc9d6f29q [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:29 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:02:29 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche5pddh to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.43/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:21 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:01:21 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche5pddh [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:59:05 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-scheduler-8f56ff78d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schelzzh5 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-scheduler-6d86bd4d9d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schelzzh5 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.36/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:17 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:18 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:50:18 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:11 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:11 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-sche4xx9l to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:12 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.33/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:12 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:12 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:12 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:14 kserve-ci-e2e-test attachdetach-controller Warning FailedAttachVolume Multi-Attach error for volume "pvc-42130d34-0182-4d20-95af-c345cf3e144c" Volume is already used by pod(s) llmisvc-model-pvc-router-manage-e8706282-kserve-7db79954c-j29rn, llmisvc-model-pvc-router-manage-e8706282-kserve-prefill-79lcxhc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:22 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:22 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:12 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-sche4xx9l [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:03:12 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvcb19f98874e050eec8ca94d49676113f0-kserve-router-scheduler-67b4bb9646 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezgh4m [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-68cc9685d6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezgh4m to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.50/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:52:01 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:31 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:54:31 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche26fhk to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:36 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.48/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:36 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:36 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:36 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:31 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:51:31 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche26fhk [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:49:35 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-749449dbc8 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-5958f4ff66-ppnlj to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:57 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.72/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:57 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:57 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:57 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:22:02 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:22:03 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:22:03 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:23:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.72:8000/health": dial tcp 10.134.0.72:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:30 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:24:46 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.72:8000/health": dial tcp 10.134.0.72:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:56 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: lora-multiple-adapters-test-kserve-5958f4ff66-ppnlj [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:56 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set lora-multiple-adapters-test-kserve-5958f4ff66 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:48 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-multiple-adapters-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-multiple-adapters-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/lora-multiple-adapters-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:56 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:57 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:22:14 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/lora-multiple-adapters-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:23:27 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [lora-multiple-adapters-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/lora-single-adapter-test-kserve-7c684b5d8f-c55z4 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.70/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:34 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:34 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:34 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:19:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.70:8000/health": dial tcp 10.134.0.70:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:16 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:31 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.70:8000/health": dial tcp 10.134.0.70:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: lora-single-adapter-test-kserve-7c684b5d8f-c55z4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set lora-single-adapter-test-kserve-7c684b5d8f from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:18 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy lora-single-adapter-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "lora-single-adapter-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/lora-single-adapter-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.HTTPRoute kserve-ci-e2e-test/lora-single-adapter-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.DestinationRule kserve-ci-e2e-test/lora-single-adapter-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:38 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/lora-single-adapter-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:19:57 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [lora-single-adapter-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-55595fd97d-6798p to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.32/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-55595fd97d-6zt4j to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.44/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulling Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:37 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" in 4.076s (4.076s including waiting). Image size: 300570044 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:38 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:38 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:45 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:01 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.44:8000/health": dial tcp 10.134.0.44:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:48:04 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.44:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: precise-prefix-cache-test-kserve-55595fd97d-6798p [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: precise-prefix-cache-test-kserve-55595fd97d-6zt4j [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: precise-prefix-cache-test-kserve-router-scheduler-5dfbc5d5jz9wh [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-5dfbc5d5jz9wh to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:34 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.33/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:34 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:34 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:34 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:35 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:36 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" in 1.697s (1.697s including waiting). Image size: 75073927 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:08 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:08 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:37 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:vllm-v0.19.1" in 30.436s (30.436s including waiting). Image size: 2989890188 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: tokenizer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container tokenizer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:08 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:14 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.132.0.33:8082/healthz": dial tcp 10.132.0.33:8082: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:25 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning Unhealthy Liveness probe failed: timeout: failed to connect service "10.132.0.33:9003" within 1s: context deadline exceeded [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container tokenizer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-5dfbc5d5c6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set precise-prefix-cache-test-kserve-55595fd97d from 0 to 2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:26 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:44 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [precise-prefix-cache-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:47:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-7cfcd8888-cb8tx to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.75/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.75:8000/health": dial tcp 10.134.0.75:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:59 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:15 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.75:8000/health": dial tcp 10.134.0.75:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-7cfcd8888-crkn4 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:59 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.60/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:04 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:04 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:04 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:19 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.132.0.60:8000/health": dial tcp 10.132.0.60:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:29 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:47 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: prestop-hook-test-kserve-7cfcd8888-cb8tx [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:59 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: prestop-hook-test-kserve-7cfcd8888-crkn4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler-8455589cb5-dhz7f to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.58/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:50 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:50 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:50 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:29 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:29 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: prestop-hook-test-kserve-router-scheduler-8455589cb5-dhz7f [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set prestop-hook-test-kserve-router-scheduler-8455589cb5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:47 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set prestop-hook-test-kserve-7cfcd8888 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/prestop-hook-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/prestop-hook-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/prestop-hook-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/prestop-hook-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/prestop-hook-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/prestop-hook-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/prestop-hook-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/prestop-hook-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/prestop-hook-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:29:58 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [prestop-hook-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:05 kserve-ci-e2e-test LLMInferenceServiceController Warning LLMInferenceServiceNotReady LLMInferenceService [prestop-hook-test] is no longer Ready because of: MainWorkloadReady, WorkloadsReady [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:29 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-prestop-hook-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-54bc76f55b-4r5j4 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.62/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:37 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:52 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: rolling-upgrade-test-kserve-54bc76f55b-4r5j4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:37 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: rolling-upgrade-test-kserve-54bc76f55b-4r5j4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-859df57bcb-vgl6t to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:26 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.42/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:26 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:26 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:26 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:03 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:18 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.133.0.42:8000/health": dial tcp 10.133.0.42:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:26 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: rolling-upgrade-test-kserve-859df57bcb-vgl6t [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-znp95 to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.41/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:03 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:03 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd-znp95 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set rolling-upgrade-test-kserve-router-scheduler-8586d4bbcd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:52 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set rolling-upgrade-test-kserve-54bc76f55b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:26 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set rolling-upgrade-test-kserve-859df57bcb from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:37 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled down replica set rolling-upgrade-test-kserve-54bc76f55b from 1 to 0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:46 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy rolling-upgrade-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "rolling-upgrade-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/rolling-upgrade-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/rolling-upgrade-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/rolling-upgrade-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:53 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/rolling-upgrade-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:08 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/rolling-upgrade-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:38 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/rolling-upgrade-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:07 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/rolling-upgrade-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:25 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [rolling-upgrade-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Deployment kserve-ci-e2e-test/rolling-upgrade-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-rolling-upgrade-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-g5nf8 to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.26/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:17 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:17 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:17 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:23 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.133.0.26:15021/healthz/ready": dial tcp 10.133.0.26:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:31 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-rqf99 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:50 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.30/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:50 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulling Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.636s (2.636s including waiting). Image size: 181251556 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:53 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:08 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:10 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning Unhealthy Readiness probe failed: HTTP probe failed with statuscode: 503 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-zxpqb to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:32 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.47/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:32 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:32 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:32 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:38 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:49 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-54c47cdfc7-rqf99 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:08 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-rqf99 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:41 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-54c47cdfc7-g5nf8 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:31 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-g5nf8 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:31 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-54c47cdfc7-zxpqb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-82xmr to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:07 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.25/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:19 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:19 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:19 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:25 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.133.0.25:15021/healthz/ready": dial tcp 10.133.0.25:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:43 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-jfcgf to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:50 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.24/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:50 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:50 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:50 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:55 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-jqhmd to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:29 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.46/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:29 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:29 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:29 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-pf5s6 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:59 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.41/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:01 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:49 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-jfcgf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:55 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-jfcgf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:07 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-82xmr [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:46:43 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-82xmr [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:55:59 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-pf5s6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:56:00 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-pf5s6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:29 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-1-openshift-default-799f46c59b-jqhmd [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:39 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-1-openshift-default-799f46c59b-jqhmd [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:29 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-1-openshift-default-799f46c59b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:49 kserve-ci-e2e-test service-controller Normal EnsuringLoadBalancer Ensuring load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:31 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-1-openshift-default-54c47cdfc7 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 16:45:54 kserve-ci-e2e-test service-controller Normal EnsuredLoadBalancer Ensured load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:39 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled down replica set router-gateway-1-openshift-default-799f46c59b from 1 to 0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:31 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled down replica set router-gateway-1-openshift-default-54c47cdfc7 from 1 to 0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:29 kserve-ci-e2e-test gateway_labeler_controller Normal AddedLabel Added label istio.io/rev=openshift-gateway to gateway router-gateway-1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-54c789bdc6-hlg2p to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:40 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.53/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:40 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:40 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:40 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:43 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:39 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-2-openshift-default-54c789bdc6-hlg2p [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:42 kserve-ci-e2e-test replicaset-controller Normal SuccessfulDelete Deleted pod: router-gateway-2-openshift-default-54c789bdc6-hlg2p [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-684dc9b4b9-xwls8 to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:40 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.35/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:54 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:54 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:54 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container istio-proxy [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:59 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Warning Unhealthy Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:40 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-gateway-2-openshift-default-684dc9b4b9-xwls8 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:39 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-2-openshift-default-54c789bdc6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:39 kserve-ci-e2e-test service-controller Normal EnsuringLoadBalancer Ensuring load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:40 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-gateway-2-openshift-default-684dc9b4b9 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:42 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled down replica set router-gateway-2-openshift-default-54c789bdc6 from 1 to 0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:43 kserve-ci-e2e-test service-controller Normal EnsuredLoadBalancer Ensured load balancer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:39 kserve-ci-e2e-test gateway_labeler_controller Normal AddedLabel Added label istio.io/rev=openshift-gateway to gateway router-gateway-2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-5894bd8c69-pc2g7 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.68/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:20:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.68:8001/health": dial tcp 10.134.0.68:8001: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:13 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:13 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-pd-test-kserve-5894bd8c69-pc2g7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-9b6cfccc6-gq2n4 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.69/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:53 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:20:47 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.69:8000/health": dial tcp 10.134.0.69:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:13 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-pd-test-kserve-prefill-9b6cfccc6-gq2n4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-pd-test-kserve-prefill-9b6cfccc6 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-pd-test-kserve-router-scheduler-5d7c74498962q4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-5d7c74498962q4 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.54/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:49 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:49 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:49 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:13 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:13 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-5d7c74498 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-pd-test-kserve-5894bd8c69 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:49 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:18:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:20:58 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [router-with-refs-pd-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:21:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-7774dd5868-nb4hp to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.59/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:06:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.59:8000/health": dial tcp 10.134.0.59:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:23 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-test-kserve-7774dd5868-nb4hp [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-5f585d4cc5-mvxgx to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.48/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:45 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:23 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:24 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: router-with-refs-test-kserve-router-scheduler-5f585d4cc5-mvxgx [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-test-kserve-router-scheduler-5f585d4cc5 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set router-with-refs-test-kserve-7774dd5868 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:42 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:43 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:44 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:04:55 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:07:08 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [router-with-refs-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:09:23 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-744bf59fbd-67hzm to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.39/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:00 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-configmap-ref-test-kserve-744bf59fbd-67hzm [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-68f87kc62h [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-68f87kc62h to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.76/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-68f876cc5d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-configmap-ref-test-kserve-744bf59fbd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:22 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:26 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:30:59 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-configmap-ref-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-5bc7bd6864-652bq to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.63/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:33 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-custom-template-test-kserve-5bc7bd6864-652bq [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler-5672wgrz to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.78/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:33 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:33 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-custom-template-test-kserve-router-scheduler-5672wgrz [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-custom-template-test-kserve-router-scheduler-567469f665 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:59 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-custom-template-test-kserve-5bc7bd6864 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:41 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-custom-template-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-custom-template-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-custom-template-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-custom-template-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-custom-template-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-custom-template-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-custom-template-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-custom-template-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-custom-template-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:13 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-custom-template-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:32 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-custom-template-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:33:33 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-custom-template-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-5c5cfcf776-bknqb to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.40/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:10 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:27 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-ha-replicas-test-kserve-5c5cfcf776-bknqb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d6msbm to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.77/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:10 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: service unhealthy (responded with "NOT_SERVING") [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:10 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:10 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85d6msbm [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcfn9h [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dcfn9h to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.61/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:10 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:10 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-6f5f85dbc4 from 0 to 2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:27 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-ha-replicas-test-kserve-5c5cfcf776 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:26 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:27 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:28 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:45 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:44 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:31:44 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:09 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-ha-replicas-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:32:10 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-7c6c58cc96-llcf6 to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.36/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:39 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-inline-config-test-kserve-7c6c58cc96-llcf6 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-inline-config-test-kserve-router-scheduler-d9ddfs65dq [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-d9ddfs65dq to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.59/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:39 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-d9ddf6c4d from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-inline-config-test-kserve-7c6c58cc96 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:05 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:10 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:10 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:38 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-inline-config-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:28:39 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.79/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:49 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.64/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:49 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.44/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:49 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:49 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:49 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-pd-migration-test-kserve-5c87756974 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:34 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-pd-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-pd-migration-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:19 kserve-ci-e2e-test LLMInferenceServiceController Warning Error Reconciliation failed: failed to reconcile workload: failed to reconcile self-signed certificates secret: failed to reconcile self-signed TLS certificate: failed to create v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs: context canceled [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:47 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:34:48 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:00 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:01 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:23 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-v06-pd-migration-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5994569d5b-9kq2t to ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.133.0.45/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulling Pulling image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Successfully pulled image "ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2" in 757ms (757ms including waiting). Image size: 44914394 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:20 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:52 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container llm-d-routing-sidecar [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:52 kserve-ci-e2e-test kubelet/ip-10-0-140-244.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-threshold-migration-test-kserve-5994569d5b-9kq2t [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-threshold-migration-test-kserve-prefill-5fc5rgxjr [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-5fc5rgxjr to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.65/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:52 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-threshold-migration-test-kserve-prefill-5fc599dc49 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-schedhc6zv to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.80/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:52 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: scheduler-v06-threshold-migration-test-kserve-router-schedhc6zv [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-threshold-migration-test-kserve-router-scheduler-dbc65f645 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set scheduler-v06-threshold-migration-test-kserve-5994569d5b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:15 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-threshold-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-threshold-migration-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:19 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:38 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:37 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:52 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [scheduler-v06-threshold-migration-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:35:52 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-54f444cf5b-ggb5d to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.64/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:17 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:25 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.64:8000/health": dial tcp 10.134.0.64:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:15 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.64:8000/health": dial tcp 10.134.0.64:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-54f444cf5b-w4hz9 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.67/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:25 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:25 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:25 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:19 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.67:8000/health": dial tcp 10.134.0.67:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:31 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:46 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Readiness probe failed: Get "https://10.134.0.67:8000/health": dial tcp 10.134.0.67:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-54f444cf5b-ggb5d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-54f444cf5b-w4hz9 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5f5d64b477-jc599 to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:20 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.52/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:20 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:20 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:20 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:21 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:21 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:21 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:31 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:31 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5f5d64b477-v9kkz to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.50/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:06 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:07 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-router-scheduler-5f5d64b477-v9kkz [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: stop-feature-test-kserve-router-scheduler-5f5d64b477-jc599 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-router-scheduler-5f5d64b477 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-router-scheduler-5f5d64b477 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-54f444cf5b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:19 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set stop-feature-test-kserve-54f444cf5b from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:14 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:04 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:11:11 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:17:30 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [stop-feature-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:13:59 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:14:01 kserve-ci-e2e-test LLMInferenceServiceController Warning LLMInferenceServiceNotReady LLMInferenceService [stop-feature-test] is no longer Ready because of: GatewaysReady, HTTPRoutesReady, InferencePoolReady, MainWorkloadReady, PrefillWorkerWorkloadReady, PrefillWorkloadReady, RouterReady, SchedulerWorkloadReady, WorkerWorkloadReady, WorkloadsReady [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-55cb9dcfdd-kvsh9 to ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.134.0.74/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:09 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Pulled Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:09 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:09 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:03 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.74:8000/health": dial tcp 10.134.0.74:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:18 kserve-ci-e2e-test kubelet/ip-10-0-129-204.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:02 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: tls-verification-test-kserve-55cb9dcfdd-kvsh9 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 None kserve-ci-e2e-test Normal Scheduled Successfully assigned kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler-74998cd6f-nbwpq to ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test multus Normal AddedInterface Add eth0 [10.132.0.57/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:7a9c075ae5e64d970cc026f71d8348f193fda0ed6de6807045e29c208821be1e" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:04 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Pulled Container image "ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:05 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:18 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Normal Killing Stopping container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:18 kserve-ci-e2e-test kubelet/ip-10-0-142-38.ec2.internal Warning FailedPreStopHook PreStopHook failed [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test replicaset-controller Normal SuccessfulCreate Created pod: tls-verification-test-kserve-router-scheduler-74998cd6f-nbwpq [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set tls-verification-test-kserve-router-scheduler-74998cd6f from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:02 kserve-ci-e2e-test deployment-controller Normal ScalingReplicaSet Scaled up replica set tls-verification-test-kserve-55cb9dcfdd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:01 kserve-ci-e2e-test OpenDataHubModelController Warning ReconcileError Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy tls-verification-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "tls-verification-test-kserve-route" not found [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:02 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/tls-verification-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ServiceAccount kserve-ci-e2e-test/tls-verification-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Role kserve-ci-e2e-test/tls-verification-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.RoleBinding kserve-ci-e2e-test/tls-verification-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Deployment kserve-ci-e2e-test/tls-verification-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:03 kserve-ci-e2e-test LLMInferenceServiceController Normal Created Created v1.Service kserve-ci-e2e-test/tls-verification-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:21 kserve-ci-e2e-test LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/tls-verification-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:20 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.Secret kserve-ci-e2e-test/tls-verification-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:25:20 kserve-ci-e2e-test LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute kserve-ci-e2e-test/tls-verification-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:13 kserve-ci-e2e-test LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [tls-verification-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:56 2026-07-08 17:27:18 kserve-ci-e2e-test LLMInferenceServiceController Normal Deleted Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-tls-verification-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:148 ### Pod scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### init-container 'llm-d-routing-sidecar' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 Flag --decoder-use-tls has been deprecated, use --enable-tls=decoder instead [e2e-llm-inference-service] Flag --prefiller-use-tls has been deprecated, use --enable-tls=prefiller instead [e2e-llm-inference-service] {"level":"info","ts":1783532088.903048,"msg":"Proxy starting","Built on":"v0.9.0-rc.2","From Git SHA":"181aa8358916e19b8844ccc752b2d6153d4b2ad6"} [e2e-llm-inference-service] {"level":"info","ts":1783532088.903113,"msg":"Proxy configuration","config":"{\"Port\":\"8000\",\"KVConnector\":\"nixlv2\",\"ECConnector\":\"\",\"DataParallelSize\":1,\"MaxIdleConnsPerHost\":1024,\"EnablePrefillerSampling\":false,\"PrefillMaxRetries\":0,\"PrefillRetryBackoff\":200000000,\"UseTLSForPrefiller\":true,\"UseTLSForDecoder\":true,\"UseTLSForEncoder\":false,\"InsecureSkipVerifyForPrefiller\":false,\"InsecureSkipVerifyForEncoder\":false,\"InsecureSkipVerifyForDecoder\":false,\"SecureServing\":true,\"CertPath\":\"/var/run/kserve/tls\",\"MooncakeBootstrapPort\":8998,\"EnableSSRFProtection\":true,\"InferencePoolNamespace\":\"kserve-ci-e2e-test\",\"InferencePoolName\":\"scheduler-v06-pd-migration-test-inference-pool\",\"PoolGroup\":\"inference.networking.x-k8s.io\",\"DecodeChunkSize\":0,\"Tracing\":false,\"MoRIIOWriteMode\":false,\"MoRIIODecodeNotifyPort\":61005,\"MoRIIODecodeHandshakePort\":6301,\"MoRIIODecodePodIP\":\"\",\"MoRIIOParallelDispatch\":false,\"MoRIIOPrefillHandshakePort\":6301,\"MoRIIOPrefillNotifyPort\":61005,\"MoRIIOTPSize\":1,\"MoRIIODPSize\":1,\"MoRIIORemoteHosts\":null,\"MoRIIODPSizeLocal\":0,\"MoRIIODecodeHosts\":null,\"DecoderURL\":\"https://localhost:8001\"}"} [e2e-llm-inference-service] {"level":"info","ts":1783532088.904078,"logger":"allowlist-validator","msg":"starting SSRF protection allowlist validator","namespace":"kserve-ci-e2e-test","poolName":"scheduler-v06-pd-migration-test-inference-pool","gvr":"inference.networking.x-k8s.io/v1alpha2, Resource=inferencepools"} [e2e-llm-inference-service] {"level":"info","ts":1783532088.9181948,"logger":"allowlist-validator","msg":"InferencePool added","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532088.9257412,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":0,"targets":{}} [e2e-llm-inference-service] {"level":"info","ts":1783532088.9258137,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":0,"targets":{}} [e2e-llm-inference-service] {"level":"info","ts":1783532089.0049355,"logger":"allowlist-validator","msg":"allowlist validator started successfully"} [e2e-llm-inference-service] {"level":"info","ts":1783532089.0060625,"logger":"proxy server on port 8000","msg":"server TLS configured"} [e2e-llm-inference-service] {"level":"info","ts":1783532089.0060775,"logger":"proxy server on port 8000","msg":"starting","addr":"[::]:8000"} [e2e-llm-inference-service] {"level":"info","ts":1783532089.4973428,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":2,"targets":{"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532089.7112045,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532090.5013728,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532098.7103677,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532098.7320254,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532098.736516,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532098.763771,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532099.5535953,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532118.9184391,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532118.9272213,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532118.927282,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:35:24 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532124.5320785,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"66c77dc9-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:35:24 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532124.5506718,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"66cab23c-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:35:28 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532128.569199,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"692fe6a5-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:35:36 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532136.5903995,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"6df78b7e-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532148.9191172,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532148.9279053,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532148.9279637,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:35:52 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532152.6130123,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"77845429-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532178.9193761,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532178.927756,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532178.9278152,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:36:24 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532184.656443,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"8a9decad-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532208.9203732,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532208.9295814,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532208.9296525,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532238.9212563,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532238.9293785,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532238.9294417,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:37:28 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532248.695947,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"b0c98717-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532268.9215128,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532268.9313216,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532268.931387,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532298.922114,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532298.9307768,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532298.9308395,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532328.9225042,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532328.931596,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532328.9316761,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532358.9235227,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532358.9315698,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532358.9316483,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:39:28 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532368.7400293,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"f856c12b-7af3-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532388.9237146,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532388.9332945,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532388.9333496,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532418.9243019,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532418.9329042,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532418.9329617,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532448.9254014,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532448.9345999,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532448.934678,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532478.9258015,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532478.9342103,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532478.9342697,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:41:28 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532488.7844963,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"3fe41b4a-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:41:33 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532493.869109,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"42ec317a-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:41:33 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532493.885889,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"42eeca9a-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:41:37 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532497.9044762,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"4553f941-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:41:45 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532505.923108,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"4a1b8231-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532508.9268985,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532508.9363687,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532508.936422,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:42:01 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532521.9531267,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"53a91fd9-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532538.9274242,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532538.936127,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532538.936185,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:42:33 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532553.9805424,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"66c0266f-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532568.9275706,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532568.935934,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532568.935995,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532598.928313,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532598.937262,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532598.937322,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:43:38 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532618.023394,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"8cec378c-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532628.9294138,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532628.937467,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532628.9375262,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532658.9297113,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532658.9383247,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532658.9383857,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532688.9302535,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532688.9392242,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532688.939283,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532718.9307945,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532718.9431093,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532718.9431698,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:45:38 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532738.0659895,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"d4796e4c-7af4-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532748.931191,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532748.9396567,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532748.9397151,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532778.9322665,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532778.941132,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532778.9411983,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532808.9327364,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532808.9463027,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532808.9463754,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532838.9333112,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532838.9433506,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532838.9434094,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:47:38 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532858.1108086,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"1c06dbfa-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:47:43 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532863.1960073,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"1f0efcc2-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:47:43 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532863.2133913,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"1f11aaf0-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:47:47 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532867.2323291,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"2176d669-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532868.9338088,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532868.9414794,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532868.9415383,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:47:55 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532875.2510643,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"263e7790-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] 2026/07/08 17:48:11 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532891.2733808,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"2fcb090c-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532898.9341998,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532898.9427862,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532898.942853,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:48:43 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532923.2969985,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"42e17b2d-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532928.9351451,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532928.9444187,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532928.9444854,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532958.9356878,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532958.9450755,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532958.9451396,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:49:47 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783532987.3398972,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"690da30d-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783532988.9367716,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783532988.9452667,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783532988.945327,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533018.937843,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783533018.9503946,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533018.950461,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533048.9386122,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783533048.947493,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533048.9475503,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533078.93882,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783533078.9480782,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533078.94815,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:51:47 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783533107.3844278,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"b09aeb63-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] {"level":"info","ts":1783533108.9390705,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783533108.9474938,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533108.9476428,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533138.9400318,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783533138.9496922,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533138.9497554,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533168.9410503,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783533168.9494412,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533168.9494987,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533198.9421206,"logger":"allowlist-validator","msg":"InferencePool updated","name":"scheduler-v06-pd-migration-test-inference-pool"} [e2e-llm-inference-service] {"level":"info","ts":1783533198.9519641,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] {"level":"info","ts":1783533198.9520528,"logger":"allowlist-validator","msg":"rebuilt allowlist","targetCount":4,"targets":{"10.132.0.64":{},"10.134.0.79":{},"scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p":{},"scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l":{}}} [e2e-llm-inference-service] 2026/07/08 17:53:47 http: proxy error: tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not 10.132.0.64 [e2e-llm-inference-service] {"level":"error","ts":1783533227.4294505,"logger":"proxy server on port 8000","msg":"prefill request failed","request_id":"f8287edc-7af5-11f1-9148-0a580a86004f","body":"","error":"prefill returned 502","stacktrace":"github.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).handleNIXLV2\n\t/workspace/pkg/sidecar/proxy/connector_nixlv2.go:242\ngithub.com/llm-d/llm-d-router/pkg/sidecar/proxy.(*Server).createRoutes.(*Server).disaggregatedPrefillHandler.func3\n\t/workspace/pkg/sidecar/proxy/chat_completions.go:183\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.(*ServeMux).ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2861\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*middleware).serveHTTP\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:178\ngo.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.NewMiddleware.func1.1\n\t/go/pkg/mod/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.68.0/handler.go:66\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2322\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:3340\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2109"} [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 I0708 17:34:49.561998 1 config.go:602] "Configuration:" =< [e2e-llm-inference-service] { [e2e-llm-inference-service] "IP": "", [e2e-llm-inference-service] "PodName": "", [e2e-llm-inference-service] "PodNameSpace": "", [e2e-llm-inference-service] "VllmDevMode": false, [e2e-llm-inference-service] "block-size": 16, [e2e-llm-inference-service] "data-parallel-rank": -1, [e2e-llm-inference-service] "data-parallel-size": 1, [e2e-llm-inference-service] "dataset-in-memory": false, [e2e-llm-inference-service] "dataset-path": "", [e2e-llm-inference-service] "dataset-table-name": "llmd", [e2e-llm-inference-service] "dataset-url": "", [e2e-llm-inference-service] "default-embedding-dimensions": 384, [e2e-llm-inference-service] "ec-transfer-config": "", [e2e-llm-inference-service] "enable-kvcache": false, [e2e-llm-inference-service] "enable-prefix-caching": false, [e2e-llm-inference-service] "enable-request-id-headers": false, [e2e-llm-inference-service] "enable-sleep-mode": false, [e2e-llm-inference-service] "enforce-eager": false, [e2e-llm-inference-service] "event-batch-size": 16, [e2e-llm-inference-service] "failure-injection-rate": 0, [e2e-llm-inference-service] "failure-types": null, [e2e-llm-inference-service] "fake-metrics": null, [e2e-llm-inference-service] "fake-metrics-refresh-interval": 100000000, [e2e-llm-inference-service] "global-cache-hit-threshold": 0, [e2e-llm-inference-service] "hash-seed": "", [e2e-llm-inference-service] "inter-token-latency": 0, [e2e-llm-inference-service] "inter-token-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-size": 1024, [e2e-llm-inference-service] "kv-cache-transfer-latency": 0, [e2e-llm-inference-service] "kv-cache-transfer-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-per-token": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-std-dev": 0, [e2e-llm-inference-service] "latency-calculator": "", [e2e-llm-inference-service] "lora-modules": null, [e2e-llm-inference-service] "max-cpu-loras": 1, [e2e-llm-inference-service] "max-loras": 1, [e2e-llm-inference-service] "max-model-len": 1024, [e2e-llm-inference-service] "max-num-seqs": 5, [e2e-llm-inference-service] "max-tool-call-array-param-length": 5, [e2e-llm-inference-service] "max-tool-call-integer-param": 100, [e2e-llm-inference-service] "max-tool-call-number-param": 100, [e2e-llm-inference-service] "max-waiting-queue-length": 1000, [e2e-llm-inference-service] "min-tool-call-array-param-length": 1, [e2e-llm-inference-service] "min-tool-call-integer-param": 0, [e2e-llm-inference-service] "min-tool-call-number-param": 0, [e2e-llm-inference-service] "mm-encoder-only": false, [e2e-llm-inference-service] "mm-processor-kwargs": "", [e2e-llm-inference-service] "mode": "random", [e2e-llm-inference-service] "model": "facebook/opt-125m", [e2e-llm-inference-service] "object-tool-call-not-required-field-probability": 50, [e2e-llm-inference-service] "port": 8001, [e2e-llm-inference-service] "prefill-overhead": 0, [e2e-llm-inference-service] "prefill-time-per-token": 0, [e2e-llm-inference-service] "prefill-time-std-dev": 0, [e2e-llm-inference-service] "seed": 1783532089561494800, [e2e-llm-inference-service] "self-signed-certs": false, [e2e-llm-inference-service] "served-model-name": [ [e2e-llm-inference-service] "facebook/opt-125m" [e2e-llm-inference-service] ], [e2e-llm-inference-service] "ssl-certfile": "/var/run/kserve/tls/tls.crt", [e2e-llm-inference-service] "ssl-keyfile": "/var/run/kserve/tls/tls.key", [e2e-llm-inference-service] "time-factor-under-load": 1, [e2e-llm-inference-service] "time-to-first-token": 0, [e2e-llm-inference-service] "time-to-first-token-std-dev": 0, [e2e-llm-inference-service] "tool-call-not-required-param-probability": 50, [e2e-llm-inference-service] "uds-socket-path": "/tmp/tokenizer/tokenizer-uds.socket", [e2e-llm-inference-service] "zmq-endpoint": "tcp://127.0.0.1:5557" [e2e-llm-inference-service] } [e2e-llm-inference-service] > [e2e-llm-inference-service] I0708 17:34:49.598787 1 tokenizer.go:104] "Model is not a real HF model, using simulated tokenizer" model="facebook/opt-125m" [e2e-llm-inference-service] I0708 17:34:49.603775 1 context.go:138] "No dataset path or URL provided, using random text for responses" [e2e-llm-inference-service] I0708 17:34:49.603853 1 communication.go:49] "Starting communication layer" [e2e-llm-inference-service] I0708 17:34:49.603898 1 simulator.go:188] "Start processing routine" [e2e-llm-inference-service] I0708 17:34:49.604135 1 http_server_tls.go:44] "HTTPS server starting with certificate files" cert="/var/run/kserve/tls/tls.crt" key="/var/run/kserve/tls/tls.key" [e2e-llm-inference-service] I0708 17:34:49.604152 1 grpc.go:126] "Server starting" protocol="gRPC" port=8001 [e2e-llm-inference-service] I0708 17:34:49.604834 1 http.go:96] "Server starting" protocol="HTTPS" port=8001 [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:148 ### Pod scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 I0708 17:34:48.920396 1 config.go:602] "Configuration:" =< [e2e-llm-inference-service] { [e2e-llm-inference-service] "IP": "", [e2e-llm-inference-service] "PodName": "", [e2e-llm-inference-service] "PodNameSpace": "", [e2e-llm-inference-service] "VllmDevMode": false, [e2e-llm-inference-service] "block-size": 16, [e2e-llm-inference-service] "data-parallel-rank": -1, [e2e-llm-inference-service] "data-parallel-size": 1, [e2e-llm-inference-service] "dataset-in-memory": false, [e2e-llm-inference-service] "dataset-path": "", [e2e-llm-inference-service] "dataset-table-name": "llmd", [e2e-llm-inference-service] "dataset-url": "", [e2e-llm-inference-service] "default-embedding-dimensions": 384, [e2e-llm-inference-service] "ec-transfer-config": "", [e2e-llm-inference-service] "enable-kvcache": false, [e2e-llm-inference-service] "enable-prefix-caching": false, [e2e-llm-inference-service] "enable-request-id-headers": false, [e2e-llm-inference-service] "enable-sleep-mode": false, [e2e-llm-inference-service] "enforce-eager": false, [e2e-llm-inference-service] "event-batch-size": 16, [e2e-llm-inference-service] "failure-injection-rate": 0, [e2e-llm-inference-service] "failure-types": null, [e2e-llm-inference-service] "fake-metrics": null, [e2e-llm-inference-service] "fake-metrics-refresh-interval": 100000000, [e2e-llm-inference-service] "global-cache-hit-threshold": 0, [e2e-llm-inference-service] "hash-seed": "", [e2e-llm-inference-service] "inter-token-latency": 0, [e2e-llm-inference-service] "inter-token-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-size": 1024, [e2e-llm-inference-service] "kv-cache-transfer-latency": 0, [e2e-llm-inference-service] "kv-cache-transfer-latency-std-dev": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-per-token": 0, [e2e-llm-inference-service] "kv-cache-transfer-time-std-dev": 0, [e2e-llm-inference-service] "latency-calculator": "", [e2e-llm-inference-service] "lora-modules": null, [e2e-llm-inference-service] "max-cpu-loras": 1, [e2e-llm-inference-service] "max-loras": 1, [e2e-llm-inference-service] "max-model-len": 1024, [e2e-llm-inference-service] "max-num-seqs": 5, [e2e-llm-inference-service] "max-tool-call-array-param-length": 5, [e2e-llm-inference-service] "max-tool-call-integer-param": 100, [e2e-llm-inference-service] "max-tool-call-number-param": 100, [e2e-llm-inference-service] "max-waiting-queue-length": 1000, [e2e-llm-inference-service] "min-tool-call-array-param-length": 1, [e2e-llm-inference-service] "min-tool-call-integer-param": 0, [e2e-llm-inference-service] "min-tool-call-number-param": 0, [e2e-llm-inference-service] "mm-encoder-only": false, [e2e-llm-inference-service] "mm-processor-kwargs": "", [e2e-llm-inference-service] "mode": "random", [e2e-llm-inference-service] "model": "facebook/opt-125m", [e2e-llm-inference-service] "object-tool-call-not-required-field-probability": 50, [e2e-llm-inference-service] "port": 8000, [e2e-llm-inference-service] "prefill-overhead": 0, [e2e-llm-inference-service] "prefill-time-per-token": 0, [e2e-llm-inference-service] "prefill-time-std-dev": 0, [e2e-llm-inference-service] "seed": 1783532088919927300, [e2e-llm-inference-service] "self-signed-certs": false, [e2e-llm-inference-service] "served-model-name": [ [e2e-llm-inference-service] "facebook/opt-125m" [e2e-llm-inference-service] ], [e2e-llm-inference-service] "ssl-certfile": "/var/run/kserve/tls/tls.crt", [e2e-llm-inference-service] "ssl-keyfile": "/var/run/kserve/tls/tls.key", [e2e-llm-inference-service] "time-factor-under-load": 1, [e2e-llm-inference-service] "time-to-first-token": 0, [e2e-llm-inference-service] "time-to-first-token-std-dev": 0, [e2e-llm-inference-service] "tool-call-not-required-param-probability": 50, [e2e-llm-inference-service] "uds-socket-path": "/tmp/tokenizer/tokenizer-uds.socket", [e2e-llm-inference-service] "zmq-endpoint": "tcp://127.0.0.1:5557" [e2e-llm-inference-service] } [e2e-llm-inference-service] > [e2e-llm-inference-service] I0708 17:34:48.958074 1 tokenizer.go:104] "Model is not a real HF model, using simulated tokenizer" model="facebook/opt-125m" [e2e-llm-inference-service] I0708 17:34:48.962884 1 context.go:138] "No dataset path or URL provided, using random text for responses" [e2e-llm-inference-service] I0708 17:34:48.962941 1 communication.go:49] "Starting communication layer" [e2e-llm-inference-service] I0708 17:34:48.962993 1 simulator.go:188] "Start processing routine" [e2e-llm-inference-service] I0708 17:34:48.963185 1 http_server_tls.go:44] "HTTPS server starting with certificate files" cert="/var/run/kserve/tls/tls.crt" key="/var/run/kserve/tls/tls.key" [e2e-llm-inference-service] I0708 17:34:48.963207 1 grpc.go:126] "Server starting" protocol="gRPC" port=8000 [e2e-llm-inference-service] I0708 17:34:48.963846 1 http.go:96] "Server starting" protocol="HTTPS" port=8000 [e2e-llm-inference-service] I0708 17:35:24.532032 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:59668\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:35:24.550644 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:59676\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:35:28.569200 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:59680\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:35:36.590523 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:45908\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:35:52.613000 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:44196\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:36:24.656448 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:48394\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:37:28.695934 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:59082\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:39:28.740006 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:53436\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:41:28.784576 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:53932\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:41:33.869211 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:47362\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:41:33.885995 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:47368\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:41:37.904605 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:47380\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:41:45.923213 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:53736\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:42:01.953247 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:58406\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:42:33.980709 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:59044\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:43:38.023520 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:42824\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:45:38.066167 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:57350\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:47:38.110947 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:40108\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:47:43.196110 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:41120\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:47:43.213504 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:41130\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:47:47.232450 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:41144\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:47:55.251187 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:38312\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:48:11.273472 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:55626\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:48:43.297060 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:41848\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:49:47.339975 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:45992\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:51:47.384456 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:41284\": remote error: tls: bad certificate" [e2e-llm-inference-service] I0708 17:53:47.429530 1 communication.go:105] "Server error" msg="error when serving connection \"10.132.0.64:8000\"<->\"10.134.0.79:45654\": remote error: tls: bad certificate" [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:148 ### Pod scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:188 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:201 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:202 "Parent": { [e2e-llm-inference-service] "TraceID": "2c865c1026b9ebdb183b80e348abdecd", [e2e-llm-inference-service] "SpanID": "3a8518e00f475638", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "SpanKind": 2, [e2e-llm-inference-service] "StartTime": "2026-07-08T17:35:52.594695791Z", [e2e-llm-inference-service] "EndTime": "2026-07-08T17:35:52.594854698Z", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "target_model", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "facebook/opt-125m" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "request_prio", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "INT64", [e2e-llm-inference-service] "Value": 0 [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "Events": null, [e2e-llm-inference-service] "Links": null, [e2e-llm-inference-service] "Status": { [e2e-llm-inference-service] "Code": "Unset", [e2e-llm-inference-service] "Description": "" [e2e-llm-inference-service] }, [e2e-llm-inference-service] "DroppedAttributes": 0, [e2e-llm-inference-service] "DroppedEvents": 0, [e2e-llm-inference-service] "DroppedLinks": 0, [e2e-llm-inference-service] "ChildSpanCount": 5, [e2e-llm-inference-service] "Resource": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.name", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "llm-d-epp" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.version", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "v0.9.0-rc.2" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "InstrumentationScope": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/requestcontrol", [e2e-llm-inference-service] "Version": "v0.9.0-rc.2", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "181aa8358916e19b8844ccc752b2d6153d4b2ad6" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] }, [e2e-llm-inference-service] "InstrumentationLibrary": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/requestcontrol", [e2e-llm-inference-service] "Version": "v0.9.0-rc.2", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "181aa8358916e19b8844ccc752b2d6153d4b2ad6" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] { [e2e-llm-inference-service] "Name": "gateway.request", [e2e-llm-inference-service] "SpanContext": { [e2e-llm-inference-service] "TraceID": "2c865c1026b9ebdb183b80e348abdecd", [e2e-llm-inference-service] "SpanID": "3a8518e00f475638", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "Parent": { [e2e-llm-inference-service] "TraceID": "00000000000000000000000000000000", [e2e-llm-inference-service] "SpanID": "0000000000000000", [e2e-llm-inference-service] "TraceFlags": "00", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "SpanKind": 2, [e2e-llm-inference-service] "StartTime": "2026-07-08T17:35:52.594627982Z", [e2e-llm-inference-service] "EndTime": "2026-07-08T17:35:52.613791732Z", [e2e-llm-inference-service] "Attributes": null, [e2e-llm-inference-service] "Events": null, [e2e-llm-inference-service] "Links": null, [e2e-llm-inference-service] "Status": { [e2e-llm-inference-service] "Code": "Unset", [e2e-llm-inference-service] "Description": "" [e2e-llm-inference-service] }, [e2e-llm-inference-service] "DroppedAttributes": 0, [e2e-llm-inference-service] "DroppedEvents": 0, [e2e-llm-inference-service] "DroppedLinks": 0, [e2e-llm-inference-service] "ChildSpanCount": 1, [e2e-llm-inference-service] "Resource": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.name", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "llm-d-epp" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.version", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "v0.9.0-rc.2" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "InstrumentationScope": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/handlers", [e2e-llm-inference-service] "Version": "v0.9.0-rc.2", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "181aa8358916e19b8844ccc752b2d6153d4b2ad6" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] }, [e2e-llm-inference-service] "InstrumentationLibrary": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/handlers", [e2e-llm-inference-service] "Version": "v0.9.0-rc.2", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "181aa8358916e19b8844ccc752b2d6153d4b2ad6" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] {"level":"info","ts":1783532184.6382911,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"5dfd7957-b399-4e53-9e85-23a4a9f5d280"} [e2e-llm-inference-service] {"level":"info","ts":1783532184.638584,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"5dfd7957-b399-4e53-9e85-23a4a9f5d280","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532248.6796162,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"ecc593bc-6838-4b7f-8aa1-c1057530a891"} [e2e-llm-inference-service] {"level":"info","ts":1783532248.6799135,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"ecc593bc-6838-4b7f-8aa1-c1057530a891","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532368.7230625,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"32c67634-3027-4c7e-a57e-77df8f470b6b"} [e2e-llm-inference-service] {"level":"info","ts":1783532368.723365,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"32c67634-3027-4c7e-a57e-77df8f470b6b","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532488.765515,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"9d2dfeb5-5534-468f-b5aa-dc2dc02c413c"} [e2e-llm-inference-service] {"level":"info","ts":1783532488.7658038,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"9d2dfeb5-5534-468f-b5aa-dc2dc02c413c","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532493.8539362,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"95777bed-9065-4bb1-b5c9-9e6cd136c11b"} [e2e-llm-inference-service] {"level":"info","ts":1783532493.8542426,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"95777bed-9065-4bb1-b5c9-9e6cd136c11b","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532493.8717918,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"bb9b691c-2f19-4517-a765-852d86b53bc2"} [e2e-llm-inference-service] {"level":"info","ts":1783532493.8720224,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"bb9b691c-2f19-4517-a765-852d86b53bc2","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532497.8892326,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"d39e8674-ac8e-4765-b83f-7ee02b8a2377"} [e2e-llm-inference-service] {"level":"info","ts":1783532497.889542,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"d39e8674-ac8e-4765-b83f-7ee02b8a2377","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532505.9082196,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"622cc6b4-2c6e-46e8-b74c-7f968169972f"} [e2e-llm-inference-service] {"level":"info","ts":1783532505.9084709,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"622cc6b4-2c6e-46e8-b74c-7f968169972f","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532521.9354863,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"62f4bc78-c04b-491a-b8a8-a93d7f7ca3a1"} [e2e-llm-inference-service] {"level":"info","ts":1783532521.9357862,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"62f4bc78-c04b-491a-b8a8-a93d7f7ca3a1","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532553.9628615,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"14652d25-e69c-4f8b-8f27-bef408519cdf"} [e2e-llm-inference-service] {"level":"info","ts":1783532553.9631813,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"14652d25-e69c-4f8b-8f27-bef408519cdf","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532618.0056758,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"6931bfa6-4e2e-47c0-bd81-000a20865237"} [e2e-llm-inference-service] {"level":"info","ts":1783532618.0059617,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"6931bfa6-4e2e-47c0-bd81-000a20865237","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532738.0482395,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"6f6f4bc4-9a43-4b89-99bc-b4caa9581bda"} [e2e-llm-inference-service] {"level":"info","ts":1783532738.048511,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"6f6f4bc4-9a43-4b89-99bc-b4caa9581bda","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532858.0941277,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"23268b55-85b4-4416-b136-9d491cee8510"} [e2e-llm-inference-service] {"level":"info","ts":1783532858.094417,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"23268b55-85b4-4416-b136-9d491cee8510","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532863.180901,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"cae46487-23ca-4bcd-af31-74d1bdafc1d1"} [e2e-llm-inference-service] {"level":"info","ts":1783532863.1811824,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"cae46487-23ca-4bcd-af31-74d1bdafc1d1","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532863.198904,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"d7bd9b7d-0623-4cd6-aa46-07821da0e63a"} [e2e-llm-inference-service] {"level":"info","ts":1783532863.199168,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"d7bd9b7d-0623-4cd6-aa46-07821da0e63a","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532867.2167714,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"b8091801-544e-4b50-a570-d7d95488ce2f"} [e2e-llm-inference-service] {"level":"info","ts":1783532867.2170615,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"b8091801-544e-4b50-a570-d7d95488ce2f","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532875.2360702,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"0b75d578-c6e1-45e1-9799-0a87eb3e23c3"} [e2e-llm-inference-service] {"level":"info","ts":1783532875.2363474,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"0b75d578-c6e1-45e1-9799-0a87eb3e23c3","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532891.2567172,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"cef88eeb-d130-46ba-9f63-25e626c91d2e"} [e2e-llm-inference-service] {"level":"info","ts":1783532891.2570202,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"cef88eeb-d130-46ba-9f63-25e626c91d2e","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532923.2805536,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"ba47abd1-35ed-42e0-a93f-85cb6565052d"} [e2e-llm-inference-service] {"level":"info","ts":1783532923.2808456,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"ba47abd1-35ed-42e0-a93f-85cb6565052d","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783532987.322922,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"3d768311-3bbf-4354-9cdf-d6dc54f5498d"} [e2e-llm-inference-service] {"level":"info","ts":1783532987.3232012,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"3d768311-3bbf-4354-9cdf-d6dc54f5498d","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783533107.3674166,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"536b0dca-74f9-4cc5-b609-046a737b987a"} [e2e-llm-inference-service] {"level":"info","ts":1783533107.3676934,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"536b0dca-74f9-4cc5-b609-046a737b987a","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1783533227.413592,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"cbd16f74-dd03-41f1-8f81-26ef62afb7f6"} [e2e-llm-inference-service] {"level":"info","ts":1783533227.4138713,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"cbd16f74-dd03-41f1-8f81-26ef62afb7f6","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 389ab7ab-c887-4117-b798-cf5a010bc51c [e2e-llm-inference-service] resourceVersion: '66420' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpoints.kubernetes.io/managed-by: endpoint-controller [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:subsets: {} [e2e-llm-inference-service] subsets: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - ip: 10.133.0.44 [e2e-llm-inference-service] nodeName: ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp [e2e-llm-inference-service] uid: 3e579365-4e21-4945-a919-fe0be337fea6 [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Endpoints [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: cdd67940-e931-47e7-a9f0-6478a16fb767 [e2e-llm-inference-service] resourceVersion: '66016' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpoints.kubernetes.io/managed-by: endpoint-controller [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:subsets: {} [e2e-llm-inference-service] subsets: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - ip: 10.132.0.64 [e2e-llm-inference-service] nodeName: ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l [e2e-llm-inference-service] uid: 9b2ee764-3a0b-4525-b254-7012fc542ecb [e2e-llm-inference-service] - ip: 10.134.0.79 [e2e-llm-inference-service] nodeName: ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p [e2e-llm-inference-service] uid: 80e3a965-265f-4108-8b60-bacd8de49f48 [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Endpoints [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-5c87756974- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 80e3a965-265f-4108-8b60-bacd8de49f48 [e2e-llm-inference-service] resourceVersion: '66015' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 5c87756974 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.134.0.79/23"],"mac_address":"0a:58:0a:86:00:4f","gateway_ips":["10.134.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.134.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.134.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.134.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.134.0.1"}],"ip_address":"10.134.0.79/23","gateway_ip":"10.134.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.134.0.79\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:86:00:4f\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-5c87756974 [e2e-llm-inference-service] uid: 6b4d2724-e4d6-4f60-841b-0c9ab366f9d5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-129-204 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"6b4d2724-e4d6-4f60-841b-0c9ab366f9d5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8001,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"llm-d-routing-sidecar"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAMESPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fieldRef: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:initContainerStatuses: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.134.0.79"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: kube-api-access-pnhfk [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/pd-sidecar [e2e-llm-inference-service] - --port=8000 [e2e-llm-inference-service] - --vllm-port=8001 [e2e-llm-inference-service] - --kv-connector=nixlv2 [e2e-llm-inference-service] - --enable-ssrf-protection=true [e2e-llm-inference-service] - --pool-group=inference.networking.x-k8s.io [e2e-llm-inference-service] - --inference-pool=kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --secure-proxy=true [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] - --decoder-use-tls=true [e2e-llm-inference-service] - --prefiller-use-tls=true [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: INFERENCE_POOL_NAMESPACE [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] - name: INFERENCE_POOL_NAME [e2e-llm-inference-service] value: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kube-api-access-pnhfk [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 10 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 5 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8001' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8001 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kube-api-access-pnhfk [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] nodeName: ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c26,c20 [e2e-llm-inference-service] fsGroup: 1000690000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-lkjqd [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-kserve-dockercfg-2s25x [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] hostIP: 10.0.129.204 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.129.204 [e2e-llm-inference-service] podIP: 10.134.0.79 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.134.0.79 [e2e-llm-inference-service] startTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] initContainerStatuses: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2 [e2e-llm-inference-service] imageID: ghcr.io/llm-d/llm-d-router-disagg-sidecar@sha256:1ec2c035216ea0a4f8a6f7ccda55398fc3d9f71f487c135f49ba0dea08163f12 [e2e-llm-inference-service] containerID: cri-o://32141100c2d66bc4667bf4368c4f5c4f10adb8cb06eb7f133fc3f7374d5e92f1 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kube-api-access-pnhfk [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] imageID: ghcr.io/llm-d/llm-d-inference-sim@sha256:bab162bd25e2ed8b15022387cdb223023aeb33be49476af9f0115c0398fb8ff5 [e2e-llm-inference-service] containerID: cri-o://057c37fe86b4da7a5cc0c84cd482a9cf0ed06db9a08722b7a6c26b79a3818c44 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kube-api-access-pnhfk [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 9b2ee764-3a0b-4525-b254-7012fc542ecb [e2e-llm-inference-service] resourceVersion: '66004' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.132.0.64/23"],"mac_address":"0a:58:0a:84:00:40","gateway_ips":["10.132.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.132.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.132.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.132.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.132.0.1"}],"ip_address":"10.132.0.64/23","gateway_ip":"10.132.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.132.0.64\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:84:00:40\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 [e2e-llm-inference-service] uid: d5a393c5-b138-4330-a4f6-064402879b77 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-142-38 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d5a393c5-b138-4330-a4f6-064402879b77"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.132.0.64"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: kube-api-access-2gwgv [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8000' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kube-api-access-2gwgv [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: File [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: default [e2e-llm-inference-service] serviceAccount: default [e2e-llm-inference-service] nodeName: ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c26,c20 [e2e-llm-inference-service] fsGroup: 1000690000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-lkjqd [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] hostIP: 10.0.142.38 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.142.38 [e2e-llm-inference-service] podIP: 10.132.0.64 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.132.0.64 [e2e-llm-inference-service] startTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] imageID: ghcr.io/llm-d/llm-d-inference-sim@sha256:bab162bd25e2ed8b15022387cdb223023aeb33be49476af9f0115c0398fb8ff5 [e2e-llm-inference-service] containerID: cri-o://920b23f2b9bb9804cf621a98bea01e1214e5917b490f6898e399f79b170f4a7f [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kube-api-access-2gwgv [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 3e579365-4e21-4945-a919-fe0be337fea6 [e2e-llm-inference-service] resourceVersion: '66419' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 59dc679cd [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.133.0.44/23"],"mac_address":"0a:58:0a:85:00:2c","gateway_ips":["10.133.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.133.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.133.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.133.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.133.0.1"}],"ip_address":"10.133.0.44/23","gateway_ip":"10.133.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.133.0.44\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:85:00:2c\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd [e2e-llm-inference-service] uid: 299b978c-f46f-4373-b2d9-7ff2e9f80969 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-140-244 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"299b978c-f46f-4373-b2d9-7ff2e9f80969"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.133.0.44"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: kube-api-access-9knqm [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - kserve-ci-e2e-test [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: disagg-headers-handler\n- type: prefill-filter\n- type: decode-filter\n\ [e2e-llm-inference-service] - type: always-disagg-pd-decider\n- parameters:\n deciders:\n prefill:\ [e2e-llm-inference-service] \ always-disagg-pd-decider\n type: disagg-profile-handler\n- type: prefix-cache-scorer\n\ [e2e-llm-inference-service] - type: queue-scorer\n- type: max-score-picker\nschedulingProfiles:\n- name:\ [e2e-llm-inference-service] \ prefill\n plugins:\n - pluginRef: prefill-filter\n - pluginRef: queue-scorer\n\ [e2e-llm-inference-service] \ weight: 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef:\ [e2e-llm-inference-service] \ max-score-picker\n- name: decode\n plugins:\n - pluginRef: decode-filter\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: prefix-cache-scorer\n\ [e2e-llm-inference-service] \ weight: 3\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] - name: kube-api-access-9knqm [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000690000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] nodeName: ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c26,c20 [e2e-llm-inference-service] fsGroup: 1000690000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-lkjqd [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-epp-sa-dockercfg-j7npf [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] hostIP: 10.0.140.244 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.140.244 [e2e-llm-inference-service] podIP: 10.133.0.44 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.133.0.44 [e2e-llm-inference-service] startTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2 [e2e-llm-inference-service] imageID: ghcr.io/llm-d/llm-d-router-endpoint-picker@sha256:06b6c75d77afd0e07053402752a9736c2dfbc12a306d0d37d963aac4c1d4e6a6 [e2e-llm-inference-service] containerID: cri-o://1575ca71ac25009c98b944b5107520ffbc3fa3bb892cd23ea1b5d543dfdb6f71 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] - name: kube-api-access-9knqm [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000690000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000690000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 7b4c6fa7-a468-465a-9c82-671fba74dbdf [e2e-llm-inference-service] resourceVersion: '65740' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] openshift.io/internal-registry-pull-secret-ref: scheduler-v06-pd-migration-test-epp-sa-dockercfg-j7npf [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: openshift.io/image-registry-pull-secrets_service-account-controller [e2e-llm-inference-service] operation: Apply [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:imagePullSecrets: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:openshift.io/internal-registry-pull-secret-ref: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] k:{"name":"scheduler-v06-pd-migration-test-epp-sa-dockercfg-j7npf"}: {} [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"default-dockercfg-lkjqd"}: {} [e2e-llm-inference-service] k:{"name":"seaweedfs-s3-creds"}: {} [e2e-llm-inference-service] secrets: [e2e-llm-inference-service] - name: default-dockercfg-lkjqd [e2e-llm-inference-service] - name: seaweedfs-s3-creds [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-epp-sa-dockercfg-j7npf [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-lkjqd [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-epp-sa-dockercfg-j7npf [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: ServiceAccount [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 9ae99a99-29a3-4bd2-b116-c60dc555df9b [e2e-llm-inference-service] resourceVersion: '65694' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:47Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] openshift.io/internal-registry-pull-secret-ref: scheduler-v06-pd-migration-test-kserve-dockercfg-2s25x [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: openshift.io/image-registry-pull-secrets_service-account-controller [e2e-llm-inference-service] operation: Apply [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:47Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:imagePullSecrets: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:openshift.io/internal-registry-pull-secret-ref: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] k:{"name":"scheduler-v06-pd-migration-test-kserve-dockercfg-2s25x"}: {} [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:47Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"default-dockercfg-lkjqd"}: {} [e2e-llm-inference-service] k:{"name":"seaweedfs-s3-creds"}: {} [e2e-llm-inference-service] secrets: [e2e-llm-inference-service] - name: default-dockercfg-lkjqd [e2e-llm-inference-service] - name: seaweedfs-s3-creds [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-kserve-dockercfg-2s25x [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-lkjqd [e2e-llm-inference-service] - name: scheduler-v06-pd-migration-test-kserve-dockercfg-2s25x [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: ServiceAccount [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 73970355-495d-4f92-a9b7-b277457e8ef5 [e2e-llm-inference-service] resourceVersion: '65763' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:internalTrafficPolicy: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"port":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:sessionAffinity: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] spec: [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] targetPort: grpc [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] targetPort: grpc-health [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] targetPort: metrics [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] targetPort: zmq [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] clusterIP: 172.31.144.242 [e2e-llm-inference-service] clusterIPs: [e2e-llm-inference-service] - 172.31.144.242 [e2e-llm-inference-service] type: ClusterIP [e2e-llm-inference-service] sessionAffinity: None [e2e-llm-inference-service] ipFamilies: [e2e-llm-inference-service] - IPv4 [e2e-llm-inference-service] ipFamilyPolicy: SingleStack [e2e-llm-inference-service] internalTrafficPolicy: Cluster [e2e-llm-inference-service] status: [e2e-llm-inference-service] loadBalancer: {} [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 2738260f-3c81-4939-9457-54be4e6b925a [e2e-llm-inference-service] resourceVersion: '65727' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:internalTrafficPolicy: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"port":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:appProtocol: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:sessionAffinity: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] spec: [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] targetPort: 8000 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] clusterIP: 172.31.173.16 [e2e-llm-inference-service] clusterIPs: [e2e-llm-inference-service] - 172.31.173.16 [e2e-llm-inference-service] type: ClusterIP [e2e-llm-inference-service] sessionAffinity: None [e2e-llm-inference-service] ipFamilies: [e2e-llm-inference-service] - IPv4 [e2e-llm-inference-service] ipFamilyPolicy: SingleStack [e2e-llm-inference-service] internalTrafficPolicy: Cluster [e2e-llm-inference-service] status: [e2e-llm-inference-service] loadBalancer: {} [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 8d6467ee-8a05-4f15-bbde-1a727586d91c [e2e-llm-inference-service] resourceVersion: '66019' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:rollingUpdate: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:maxSurge: {} [e2e-llm-inference-service] f:maxUnavailable: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8001,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"llm-d-routing-sidecar"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAMESPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fieldRef: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/pd-sidecar [e2e-llm-inference-service] - --port=8000 [e2e-llm-inference-service] - --vllm-port=8001 [e2e-llm-inference-service] - --kv-connector=nixlv2 [e2e-llm-inference-service] - --enable-ssrf-protection=true [e2e-llm-inference-service] - --pool-group=inference.networking.x-k8s.io [e2e-llm-inference-service] - --inference-pool=kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --secure-proxy=true [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] - --decoder-use-tls=true [e2e-llm-inference-service] - --prefiller-use-tls=true [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: INFERENCE_POOL_NAMESPACE [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] - name: INFERENCE_POOL_NAME [e2e-llm-inference-service] value: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 10 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 5 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8001' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8001 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: RollingUpdate [e2e-llm-inference-service] rollingUpdate: [e2e-llm-inference-service] maxUnavailable: 25% [e2e-llm-inference-service] maxSurge: 25% [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "scheduler-v06-pd-migration-test-kserve-5c87756974" has successfully [e2e-llm-inference-service] progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 14755bae-8c99-4fb3-9ffd-15e5a9e1e33c [e2e-llm-inference-service] resourceVersion: '66008' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:rollingUpdate: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:maxSurge: {} [e2e-llm-inference-service] f:maxUnavailable: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8000' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: File [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: RollingUpdate [e2e-llm-inference-service] rollingUpdate: [e2e-llm-inference-service] maxUnavailable: 25% [e2e-llm-inference-service] maxSurge: 25% [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847" [e2e-llm-inference-service] has successfully progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: b165e08b-b94a-4a2a-9e21-668c887da1ea [e2e-llm-inference-service] resourceVersion: '66423' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - kserve-ci-e2e-test [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: disagg-headers-handler\n- type: prefill-filter\n- type:\ [e2e-llm-inference-service] \ decode-filter\n- type: always-disagg-pd-decider\n- parameters:\n deciders:\n\ [e2e-llm-inference-service] \ prefill: always-disagg-pd-decider\n type: disagg-profile-handler\n\ [e2e-llm-inference-service] - type: prefix-cache-scorer\n- type: queue-scorer\n- type: max-score-picker\n\ [e2e-llm-inference-service] schedulingProfiles:\n- name: prefill\n plugins:\n - pluginRef: prefill-filter\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: prefix-cache-scorer\n\ [e2e-llm-inference-service] \ weight: 3\n - pluginRef: max-score-picker\n- name: decode\n plugins:\n\ [e2e-llm-inference-service] \ - pluginRef: decode-filter\n - pluginRef: queue-scorer\n weight:\ [e2e-llm-inference-service] \ 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: Recreate [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd" [e2e-llm-inference-service] has successfully progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-5c87756974 [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 6b4d2724-e4d6-4f60-841b-0c9ab366f9d5 [e2e-llm-inference-service] resourceVersion: '66018' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 5c87756974 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '2' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] uid: 8d6467ee-8a05-4f15-bbde-1a727586d91c [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"8d6467ee-8a05-4f15-bbde-1a727586d91c"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8001,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"llm-d-routing-sidecar"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"INFERENCE_POOL_NAMESPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fieldRef: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 5c87756974 [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 5c87756974 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-disagg-sidecar:v0.9.0-rc.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/pd-sidecar [e2e-llm-inference-service] - --port=8000 [e2e-llm-inference-service] - --vllm-port=8001 [e2e-llm-inference-service] - --kv-connector=nixlv2 [e2e-llm-inference-service] - --enable-ssrf-protection=true [e2e-llm-inference-service] - --pool-group=inference.networking.x-k8s.io [e2e-llm-inference-service] - --inference-pool=kserve-ci-e2e-test/scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --secure-proxy=true [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] - --decoder-use-tls=true [e2e-llm-inference-service] - --prefiller-use-tls=true [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: INFERENCE_POOL_NAMESPACE [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] - name: INFERENCE_POOL_NAME [e2e-llm-inference-service] value: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] resources: {} [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 10 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] initialDelaySeconds: 10 [e2e-llm-inference-service] timeoutSeconds: 5 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8001' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8001 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8001 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847 [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: d5a393c5-b138-4330-a4f6-064402879b77 [e2e-llm-inference-service] resourceVersion: '66007' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '2' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill [e2e-llm-inference-service] uid: 14755bae-8c99-4fb3-9ffd-15e5a9e1e33c [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"14755bae-8c99-4fb3-9ffd-15e5a9e1e33c"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:58Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-inference-sim:v0.8.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/llm-d-inference-sim [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --port [e2e-llm-inference-service] - '8000' [e2e-llm-inference-service] - --model [e2e-llm-inference-service] - facebook/opt-125m [e2e-llm-inference-service] - --mode [e2e-llm-inference-service] - random [e2e-llm-inference-service] - --ssl-certfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.crt [e2e-llm-inference-service] - --ssl-keyfile [e2e-llm-inference-service] - /var/run/kserve/tls/tls.key [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: INFO [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: File [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-59dc679cd [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 299b978c-f46f-4373-b2d9-7ff2e9f80969 [e2e-llm-inference-service] resourceVersion: '66422' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 59dc679cd [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler [e2e-llm-inference-service] uid: b165e08b-b94a-4a2a-9e21-668c887da1ea [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"b165e08b-b94a-4a2a-9e21-668c887da1ea"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 59dc679cd [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 59dc679cd [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: scheduler-v06-pd-migration-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: ghcr.io/llm-d/llm-d-router-endpoint-picker:v0.9.0-rc.2 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - kserve-ci-e2e-test [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: disagg-headers-handler\n- type: prefill-filter\n- type:\ [e2e-llm-inference-service] \ decode-filter\n- type: always-disagg-pd-decider\n- parameters:\n deciders:\n\ [e2e-llm-inference-service] \ prefill: always-disagg-pd-decider\n type: disagg-profile-handler\n\ [e2e-llm-inference-service] - type: prefix-cache-scorer\n- type: queue-scorer\n- type: max-score-picker\n\ [e2e-llm-inference-service] schedulingProfiles:\n- name: prefill\n plugins:\n - pluginRef: prefill-filter\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: prefix-cache-scorer\n\ [e2e-llm-inference-service] \ weight: 3\n - pluginRef: max-score-picker\n- name: decode\n plugins:\n\ [e2e-llm-inference-service] \ - pluginRef: decode-filter\n - pluginRef: queue-scorer\n weight:\ [e2e-llm-inference-service] \ 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] serviceAccount: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 7bc6032b-53f9-4157-bd45-18b9056caf86 [e2e-llm-inference-service] resourceVersion: '65757' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] apiGroup: rbac.authorization.k8s.io [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 0fd4cb7b-a0cc-48e6-bdf8-5daf28ff4124 [e2e-llm-inference-service] resourceVersion: '65707' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] apiGroup: rbac.authorization.k8s.io [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 17220c81-2cd3-4df4-9461-8381991898f2 [e2e-llm-inference-service] resourceVersion: '65754' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - inferencemodels [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - inferencepoolimports [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - discovery.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - endpointslices [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] - create [e2e-llm-inference-service] - update [e2e-llm-inference-service] - patch [e2e-llm-inference-service] - delete [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - coordination.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - leases [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: d6e0a4bc-041d-49f9-ad77-e96bbf76594a [e2e-llm-inference-service] resourceVersion: '65702' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service-wrd64 [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-epp-service- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: dbca4f10-83a2-4eac-ad60-12df1ab1ac23 [e2e-llm-inference-service] resourceVersion: '66421' [e2e-llm-inference-service] generation: 3 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io [e2e-llm-inference-service] kubernetes.io/service-name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] uid: 73970355-495d-4f92-a9b7-b277457e8ef5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:35:20Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:addressType: {} [e2e-llm-inference-service] f:endpoints: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpointslice.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:kubernetes.io/service-name: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"73970355-495d-4f92-a9b7-b277457e8ef5"}: {} [e2e-llm-inference-service] f:ports: {} [e2e-llm-inference-service] addressType: IPv4 [e2e-llm-inference-service] endpoints: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.133.0.44 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp [e2e-llm-inference-service] uid: 3e579365-4e21-4945-a919-fe0be337fea6 [e2e-llm-inference-service] nodeName: ip-10-0-140-244.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] kind: EndpointSlice [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc-4sg2h [e2e-llm-inference-service] generateName: scheduler-v06-pd-migration-test-kserve-workload-svc- [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: f369d0a6-2914-40d6-bdeb-4fefa9be3b22 [e2e-llm-inference-service] resourceVersion: '66017' [e2e-llm-inference-service] generation: 5 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io [e2e-llm-inference-service] kubernetes.io/service-name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] uid: 2738260f-3c81-4939-9457-54be4e6b925a [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:59Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:addressType: {} [e2e-llm-inference-service] f:endpoints: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpointslice.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:kubernetes.io/service-name: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"2738260f-3c81-4939-9457-54be4e6b925a"}: {} [e2e-llm-inference-service] f:ports: {} [e2e-llm-inference-service] addressType: IPv4 [e2e-llm-inference-service] endpoints: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.134.0.79 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p [e2e-llm-inference-service] uid: 80e3a965-265f-4108-8b60-bacd8de49f48 [e2e-llm-inference-service] nodeName: ip-10-0-129-204.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.132.0.64 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l [e2e-llm-inference-service] uid: 9b2ee764-3a0b-4525-b254-7012fc542ecb [e2e-llm-inference-service] nodeName: ip-10-0-142-38.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] kind: EndpointSlice [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 7bc6032b-53f9-4157-bd45-18b9056caf86 [e2e-llm-inference-service] resourceVersion: '65757' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] userNames: [e2e-llm-inference-service] - system:serviceaccount:kserve-ci-e2e-test:scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] groupNames: null [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-sa [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-rb [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 0fd4cb7b-a0cc-48e6-bdf8-5daf28ff4124 [e2e-llm-inference-service] resourceVersion: '65707' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] userNames: [e2e-llm-inference-service] - system:serviceaccount:kserve-ci-e2e-test:scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] groupNames: null [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: 17220c81-2cd3-4df4-9461-8381991898f2 [e2e-llm-inference-service] resourceVersion: '65754' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodels [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - inferencepoolimports [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - discovery.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - endpointslices [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - create [e2e-llm-inference-service] - delete [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - patch [e2e-llm-inference-service] - update [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - coordination.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - leases [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-role [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] uid: d6e0a4bc-041d-49f9-ad77-e96bbf76594a [e2e-llm-inference-service] resourceVersion: '65702' [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/inference-pool-migrated: v1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] generation: 2 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/inference-pool-migrated: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '66050' [e2e-llm-inference-service] uid: 45c16804-369b-4097-acdf-d84b68308fa5 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] parentRefs: [e2e-llm-inference-service] - group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: / [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] message: Route was valid [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] message: All references resolved [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] controllerName: openshift.io/gateway-controller/v1 [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] message: Object affected by AuthPolicy [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route-authn [e2e-llm-inference-service] openshift-ingress/openshift-ai-inference-authn] [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: kuadrant.io/AuthPolicyAffected [e2e-llm-inference-service] controllerName: kuadrant.io/policy-controller [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/inference-pool-migrated: v1 [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] generation: 2 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/inference-pool-migrated: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '66050' [e2e-llm-inference-service] uid: 45c16804-369b-4097-acdf-d84b68308fa5 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] parentRefs: [e2e-llm-inference-service] - group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /kserve-ci-e2e-test/scheduler-v06-pd-migration-test [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/kserve-ci-e2e-test/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: / [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] message: Route was valid [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] message: All references resolved [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] controllerName: openshift.io/gateway-controller/v1 [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] message: Object affected by AuthPolicy [kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route-authn [e2e-llm-inference-service] openshift-ingress/openshift-ai-inference-authn] [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: kuadrant.io/AuthPolicyAffected [e2e-llm-inference-service] controllerName: kuadrant.io/policy-controller [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:appProtocol: {} [e2e-llm-inference-service] f:endpointPickerRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureMode: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:number: {} [e2e-llm-inference-service] f:selector: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:matchLabels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:targetPorts: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] - apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '66038' [e2e-llm-inference-service] uid: dccbe2b9-6c1e-4a55-b539-76c11d0cda3c [e2e-llm-inference-service] spec: [e2e-llm-inference-service] appProtocol: http [e2e-llm-inference-service] endpointPickerRef: [e2e-llm-inference-service] failureMode: FailOpen [e2e-llm-inference-service] group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] port: [e2e-llm-inference-service] number: 9002 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] targetPorts: [e2e-llm-inference-service] - number: 8000 [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] message: Referenced by an HTTPRoute accepted by the parentRef Gateway [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] message: Referenced ExtensionRef resolved successfully [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: networking.istio.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: kuadrant.io/v1 [e2e-llm-inference-service] kind: AuthPolicy [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-policies [e2e-llm-inference-service] app.kubernetes.io/managed-by: odh-model-controller [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: kuadrant.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:rules: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:authentication: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:public: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:anonymous: {} [e2e-llm-inference-service] f:credentials: {} [e2e-llm-inference-service] f:metrics: {} [e2e-llm-inference-service] f:overrides: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:fairness: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:objective: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:priority: {} [e2e-llm-inference-service] f:response: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:success: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:headers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:x-gateway-inference-fairness-id: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:metrics: {} [e2e-llm-inference-service] f:plain: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:expression: {} [e2e-llm-inference-service] f:priority: {} [e2e-llm-inference-service] f:x-gateway-inference-objective: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:metrics: {} [e2e-llm-inference-service] f:plain: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:expression: {} [e2e-llm-inference-service] f:priority: {} [e2e-llm-inference-service] f:targetRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] - apiVersion: kuadrant.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Accepted"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Enforced"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-08T17:34:51Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route-authn [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65924' [e2e-llm-inference-service] uid: 0cd3eeca-631b-4617-aad1-fbd3fe9e9ba8 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] rules: [e2e-llm-inference-service] authentication: [e2e-llm-inference-service] public: [e2e-llm-inference-service] anonymous: {} [e2e-llm-inference-service] credentials: {} [e2e-llm-inference-service] metrics: false [e2e-llm-inference-service] overrides: [e2e-llm-inference-service] fairness: [e2e-llm-inference-service] value: unauthenticated [e2e-llm-inference-service] objective: [e2e-llm-inference-service] value: unauthenticated [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] response: [e2e-llm-inference-service] success: [e2e-llm-inference-service] headers: [e2e-llm-inference-service] x-gateway-inference-fairness-id: [e2e-llm-inference-service] metrics: false [e2e-llm-inference-service] plain: [e2e-llm-inference-service] expression: auth.identity.fairness [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] x-gateway-inference-objective: [e2e-llm-inference-service] metrics: false [e2e-llm-inference-service] plain: [e2e-llm-inference-service] expression: auth.identity.objective [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-route [e2e-llm-inference-service] status: [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:34:51Z' [e2e-llm-inference-service] message: AuthPolicy has been accepted [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-08T17:34:51Z' [e2e-llm-inference-service] message: AuthPolicy has been successfully enforced [e2e-llm-inference-service] reason: Enforced [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Enforced [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65857' [e2e-llm-inference-service] uid: 4df0d16f-b9e7-40ca-b473-9b4ab9a3d836 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '66042' [e2e-llm-inference-service] uid: 54e503df-130b-4866-854c-92e26d0eeb92 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-inference-pool-ip-2bbd07cf.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65863' [e2e-llm-inference-service] uid: 0a50dbcc-150b-42cf-9d49-92ebaa3e6ac6 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65857' [e2e-llm-inference-service] uid: 4df0d16f-b9e7-40ca-b473-9b4ab9a3d836 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '66042' [e2e-llm-inference-service] uid: 54e503df-130b-4866-854c-92e26d0eeb92 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-inference-pool-ip-2bbd07cf.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65863' [e2e-llm-inference-service] uid: 0a50dbcc-150b-42cf-9d49-92ebaa3e6ac6 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:49Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-scheduler [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65857' [e2e-llm-inference-service] uid: 4df0d16f-b9e7-40ca-b473-9b4ab9a3d836 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-epp-service.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:35:01Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '66042' [e2e-llm-inference-service] uid: 54e503df-130b-4866-854c-92e26d0eeb92 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-inference-pool-ip-2bbd07cf.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:50Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-workload-svc [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65863' [e2e-llm-inference-service] uid: 0a50dbcc-150b-42cf-9d49-92ebaa3e6ac6 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: scheduler-v06-pd-migration-test-kserve-workload-svc.kserve-ci-e2e-test.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 apiVersion: inference.networking.x-k8s.io/v1alpha2 [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: inference.networking.x-k8s.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"e96687d2-b656-4a14-964b-4f19f4089bae"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:extensionRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureMode: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:portNumber: {} [e2e-llm-inference-service] f:selector: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:targetPortNumber: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-08T17:34:48Z' [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-inference-pool [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] uid: e96687d2-b656-4a14-964b-4f19f4089bae [e2e-llm-inference-service] resourceVersion: '65781' [e2e-llm-inference-service] uid: bb0414dd-de66-41d5-a968-012f374cae52 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] extensionRef: [e2e-llm-inference-service] failureMode: FailOpen [e2e-llm-inference-service] group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-epp-service [e2e-llm-inference-service] portNumber: 9002 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] targetPortNumber: 8000 [e2e-llm-inference-service] status: [e2e-llm-inference-service] parent: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '1970-01-01T00:00:00Z' [e2e-llm-inference-service] message: Waiting for controller [e2e-llm-inference-service] reason: Pending [e2e-llm-inference-service] status: Unknown [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Status [e2e-llm-inference-service] name: default [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:53:49Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: decode [e2e-llm-inference-service] pod-template-hash: 5c87756974 [e2e-llm-inference-service] timestamp: '2026-07-08T17:53:33Z' [e2e-llm-inference-service] window: 17.521s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: llm-d-routing-sidecar [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 12722611n [e2e-llm-inference-service] memory: 17660Ki [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 19623936n [e2e-llm-inference-service] memory: 25132Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:53:49Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload-prefill [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: prefill [e2e-llm-inference-service] pod-template-hash: 78b7f4c847 [e2e-llm-inference-service] timestamp: '2026-07-08T17:53:30Z' [e2e-llm-inference-service] window: 12.335s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 18774706n [e2e-llm-inference-service] memory: 26756Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1254 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1255 metadata: [e2e-llm-inference-service] name: scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp [e2e-llm-inference-service] namespace: kserve-ci-e2e-test [e2e-llm-inference-service] creationTimestamp: '2026-07-08T17:53:49Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: scheduler-v06-pd-migration-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 59dc679cd [e2e-llm-inference-service] timestamp: '2026-07-08T17:53:34Z' [e2e-llm-inference-service] window: 15.511s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 14758816n [e2e-llm-inference-service] memory: 30572Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_inference_service] [2026-07-08T17:53:49.741941] end - ❌ 1196.227s: Service returned 502: [e2e-llm-inference-service] =============================== warnings summary =============================== [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] /workspace/source/python/kserve/.venv/lib64/python3.11/site-packages/pytest_asyncio/plugin.py:761: DeprecationWarning: The event_loop fixture provided by pytest-asyncio has been redefined in [e2e-llm-inference-service] /workspace/source/test/e2e/conftest.py:43 [e2e-llm-inference-service] Replacing the event_loop fixture with a custom implementation is deprecated [e2e-llm-inference-service] and will lead to errors in the future. [e2e-llm-inference-service] If you want to request an asyncio event loop with a scope other than function [e2e-llm-inference-service] scope, use the "scope" argument to the asyncio mark when marking the tests. [e2e-llm-inference-service] If you want to return different types of event loops, use the event_loop_policy [e2e-llm-inference-service] fixture. [e2e-llm-inference-service] [e2e-llm-inference-service] warnings.warn( [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-gateway-ref-router-with-managed-route-model-fb-opt-125m-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py:40: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_tls.py:93: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:245: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html [e2e-llm-inference-service] ---------- generated xml file: /workspace/artifacts-dir/junit_e2e.xml ---------- [e2e-llm-inference-service] --------------------------------- JSON report ---------------------------------- [e2e-llm-inference-service] report saved to: /workspace/artifacts-dir/e2e_results.json [e2e-llm-inference-service] =========================== short test summary info ============================ [e2e-llm-inference-service] FAILED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] ====== 1 failed, 41 passed, 3 skipped, 29 warnings in 4083.19s (1:08:03) ======= [must-gather] [must-gather ] OUT 2026-07-08T17:53:51.036783474Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: d1dd22ee-2e29-42b5-ba81-0a2594b31f7b [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.23" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-08T17:53:51.071585183Z namespace/openshift-must-gather-5rg2q created [must-gather] [must-gather ] OUT 2026-07-08T17:53:51.078679947Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-6c4cc created [must-gather] [must-gather ] OUT 2026-07-08T17:53:51.103333635Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:57.871271979Z [disk usage checker] Started [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:57.874670644Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.149343702Z Error from server (NotFound): namespaces "redhat-ods-operator" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.328336614Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.328368298Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.328368298Z namespaces "redhat-ods-operator" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.331574626Z Error getting logs from redhat-ods-operator [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.506589060Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.506621015Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.506621015Z namespaces "redhat-ods-monitoring" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.509416758Z Error getting logs from redhat-ods-monitoring [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.679867789Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.679904847Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.679904847Z namespaces "redhat-ods-applications" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.682757267Z Error getting logs from redhat-ods-applications [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.854291647Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.854403338Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.854403338Z namespaces "rhods-notebooks" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:58.857448154Z Error getting logs from rhods-notebooks [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.025132067Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.025234725Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.025234725Z namespaces "rhoai-model-registries" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.028029210Z Error getting logs from rhoai-model-registries [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.198762575Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.198861098Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.198861098Z namespaces "istio-system" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.201719754Z Error getting logs from istio-system [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.374397320Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.374429305Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.374429305Z namespaces "knative-serving" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.377070771Z Error getting logs from knative-serving [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.549068611Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.549098760Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.549098760Z namespaces "redhat-ods-applications-auth-provider" not found [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.551992095Z Error getting logs from redhat-ods-applications-auth-provider [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:53:59.844848758Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.145509508Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.300940094Z error: the server doesn't have a resource type "auths" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.388060776Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.390565509Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.536898827Z error: the server doesn't have a resource type "monitorings" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.627533095Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.630415473Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.768700205Z error: the server doesn't have a resource type "featuretrackers" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.853343989Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.856200254Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:00.993548755Z error: the server doesn't have a resource type "codeflares" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.080520622Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.083099363Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.222960234Z error: the server doesn't have a resource type "dashboards" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.306397242Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.309026799Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.445892807Z error: the server doesn't have a resource type "datasciencepipelines" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.532230489Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.534851518Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.673512116Z error: the server doesn't have a resource type "feastoperators" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.762659831Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.765938217Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.903831728Z error: the server doesn't have a resource type "kserves" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.989813121Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:01.992785934Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.136089487Z error: the server doesn't have a resource type "kueues" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.221229658Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.223961011Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.360239836Z error: the server doesn't have a resource type "modelcontrollers" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.447087838Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.449829257Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.595732956Z error: the server doesn't have a resource type "modelmeshservings" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.684246113Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.687247108Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.824465442Z error: the server doesn't have a resource type "modelregistries" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.879174537Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.915886623Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:02.918706010Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.062906745Z error: the server doesn't have a resource type "rays" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.148040495Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.150701700Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.291503424Z error: the server doesn't have a resource type "trainingoperators" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.380087789Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.382882179Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.524739007Z error: the server doesn't have a resource type "trustyais" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.615068609Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.617719417Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.761034603Z error: the server doesn't have a resource type "workbenches" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.847297777Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.850298517Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:03.996386182Z error: the server doesn't have a resource type "hardwareprofiles" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:04.086940859Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:04.089972533Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:04.236007688Z error: the server doesn't have a resource type "llamastackoperators" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:04.326319375Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:04.329219389Z Error collecting info from [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:05.093834561Z error: the server doesn't have a resource type "predictors" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:05.237694818Z error: the server doesn't have a resource type "localmodelnodegroups" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:05.391154218Z error: the server doesn't have a resource type "smcp" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:05.530434570Z error: the server doesn't have a resource type "smm" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:05.670125344Z error: the server doesn't have a resource type "smmr" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:06.314958774Z error: the server doesn't have a resource type "knativeservings" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:06.460216394Z error: the server doesn't have a resource type "configurations" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:06.607424126Z error: the server doesn't have a resource type "routes" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:06.755114564Z error: the server doesn't have a resource type "services" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:06.894749186Z error: the server doesn't have a resource type "revisions" [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:07.480249079Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:07.838818734Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:07.883825453Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:08.586268784Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:08.969615611Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:09.152914859Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:09.480270253Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:10.258759459Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:10.636847608Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:10.805731542Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:11.114299785Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:11.845342388Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:12.200278783Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:12.382402109Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:12.693269727Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:12.888330736Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:13.423939916Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:13.824088674Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:13.999331255Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:14.426055336Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:15.145792689Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:15.522831592Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:15.698958089Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:16.019368210Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:16.785867512Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:17.173616912Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:17.357680958Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:17.676480962Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:17.892848369Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:18.442992896Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:18.822461901Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:19.005695333Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:19.316188667Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:20.132186103Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:20.516934586Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:20.736969034Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:21.038100568Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:21.762180118Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:22.131193209Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:22.301531048Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:22.626621704Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:22.897639229Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:23.364128187Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:23.756208954Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:23.931386405Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:24.238492887Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:24.965420090Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:25.315728283Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:25.492163265Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:25.789333354Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:26.524275281Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:26.890698991Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:27.070399915Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:27.373937482Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:27.902213770Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:28.099453761Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:28.485912152Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:28.674145200Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:28.970666979Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:29.715164122Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:30.068451928Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:30.247369076Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:30.543732031Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:31.283820389Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:31.655424465Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:31.832100549Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:32.124896971Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:32.862769238Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:32.907821579Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:33.234260635Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:33.404751717Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:34.046165281Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:34.215593324Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:34.612151066Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:34.785873332Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:35.442039268Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:35.621011166Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:36.012473211Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:36.190114810Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:36.839902060Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:37.014665281Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:37.415987206Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:37.590087033Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:37.913484770Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:38.254818429Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:38.420441670Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:38.818003756Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:38.999436584Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:39.504505802Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:39.760923291Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:40.173460514Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:40.348703605Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:40.816250249Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:41.025253940Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:41.438913588Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:41.617248141Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:42.064101550Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:42.271411713Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:42.702002435Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:42.877537141Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:42.919458346Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:43.332455100Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:43.524182534Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:43.948549324Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:44.118117470Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:44.567163355Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:44.763086052Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:45.169260921Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:45.341081492Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:45.771205652Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:45.961391584Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:46.370157837Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:46.549435671Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:46.991146159Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:47.191038754Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:47.600236588Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:47.773670527Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:47.924318271Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:48.221276619Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:48.416417693Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:48.827666658Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:49.002378030Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:49.456471489Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:49.643001973Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:50.053381653Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:50.229084619Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:50.708800554Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:50.919571416Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:51.329485689Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:51.500660647Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:51.944120159Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:52.142021238Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:52.569702030Z Wrote inspect data to must-gather. [must-gather] [must-gather-8xrfv] POD 2026-07-08T17:54:52.594784038Z Caches written to disk [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.604072744Z waiting for gather to complete [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.6067295Z downloading gather output [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.934904729Z receiving incremental file list [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.949415994Z ./ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.949494946Z aggregated-discovery-api.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.991266361Z aggregated-discovery-apis.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.992201968Z event-filter.html [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.994784112Z timestamp [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.994966997Z version [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995046609Z cluster-scoped-resources/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.99507686Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995084201Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995096381Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995176303Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995193064Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995221814Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995264516Z namespaces/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995272736Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995337398Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.99540966Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995445971Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995482772Z namespaces/kserve-ci-e2e-test/apps/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995537463Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995617536Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.995973216Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996371277Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996422479Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.99648211Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996523832Z namespaces/kserve-ci-e2e-test/batch/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996582903Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996682516Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996752228Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.9968073Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996934763Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.996985835Z namespaces/kserve-ci-e2e-test/core/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.997037096Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.997185541Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:53.997315244Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.002460372Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.002549674Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.002908484Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003029268Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003296025Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003382428Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003426749Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003510982Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003563293Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003626885Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003648126Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003746848Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003779269Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003852301Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003911203Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003942144Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.003995115Z namespaces/kserve-ci-e2e-test/pods/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004004506Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004039587Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/e2e-pvc-model-download-lnwcc.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004109709Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/storage-initializer/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004119769Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/storage-initializer/storage-initializer/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004126979Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/storage-initializer/storage-initializer/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004135129Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/storage-initializer/storage-initializer/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004275704Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/storage-initializer/storage-initializer/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004349615Z namespaces/kserve-ci-e2e-test/pods/e2e-pvc-model-download-lnwcc/storage-initializer/storage-initializer/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004370846Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004420957Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/router-gateway-1-openshift-default-54c47cdfc7-zxpqb.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.00451864Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/istio-proxy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004528881Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/istio-proxy/istio-proxy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004533841Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004558881Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004688025Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/istio-proxy/istio-proxy/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004759397Z namespaces/kserve-ci-e2e-test/pods/router-gateway-1-openshift-default-54c47cdfc7-zxpqb/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004776798Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-xwls8/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.00485173Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-xwls8/router-gateway-2-openshift-default-684dc9b4b9-xwls8.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004938352Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-xwls8/istio-proxy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004949673Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-xwls8/istio-proxy/istio-proxy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004954333Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-xwls8/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.004995754Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-xwls8/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005102517Z namespaces/kserve-ci-e2e-test/pods/router-gateway-2-openshift-default-684dc9b4b9-xwls8/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005182509Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005227681Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005318273Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/llm-d-routing-sidecar/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005327563Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/llm-d-routing-sidecar/llm-d-routing-sidecar/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005331684Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005357974Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005604551Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005675493Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/llm-d-routing-sidecar/llm-d-routing-sidecar/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005693014Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/main/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005718485Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/main/main/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005722915Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/main/main/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005759306Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/main/main/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.00589734Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/main/main/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005968742Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-5c87756974-k4l7p/main/main/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.005984662Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006044614Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006123806Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/main/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006134597Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/main/main/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006139137Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/main/main/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006172428Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/main/main/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006287371Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/main/main/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006353743Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-prefill-78b7f4c847-bgb8l/main/main/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006389904Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006436065Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006518597Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/main/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006527698Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/main/main/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006532418Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/main/main/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006553898Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/main/main/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006825476Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/main/main/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.006900668Z namespaces/kserve-ci-e2e-test/pods/scheduler-v06-pd-migration-test-kserve-router-scheduler-59knzjp/main/main/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.00693961Z namespaces/kserve-ci-e2e-test/policy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.00696829Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007025932Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007047253Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007098324Z namespaces/kuadrant-system/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007144886Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007206867Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007234168Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.00729565Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007344371Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007430454Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.007731412Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008019451Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008069082Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008099673Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008137204Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008180255Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008264087Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008311079Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.00835197Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008430882Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008481014Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008516715Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008668199Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.008799173Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.009212245Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.009290957Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.009693228Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.009812942Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010038678Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010164942Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010195303Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010312206Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010342077Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010380368Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.01043569Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010518012Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010562393Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010585434Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010630875Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010671536Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010715168Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010730778Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010757219Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino-7c77957575-ddrbt.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010857422Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010871752Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010880173Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.010896383Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014193227Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014261669Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.01427582Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014331611Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/authorino-operator-6d85f6564-k7t49.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014434254Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014443294Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014448324Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014457595Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.01465195Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014716522Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014750833Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014809505Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/dns-operator-controller-manager-65b49595d7-cklsz.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014931108Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014941189Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.014946149Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.01498746Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015112603Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015196696Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015217446Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015294389Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin-85fd4b8b9d-cpc85.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015379761Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015390582Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015407402Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015443373Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015524495Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015612258Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015656319Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.0157054Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015852685Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015866895Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015872705Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.015881516Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.022620428Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.02268854Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.022708331Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.022765462Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador-limitador-69574b596d-m97zt.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.022870045Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.022883816Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.022892076Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.022916327Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023004849Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023072661Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023092272Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023150853Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/limitador-operator-controller-manager-6f9f468797-4hm5n.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023235956Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023248586Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023266617Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023294837Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023429032Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023491913Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023534675Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023554325Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023614027Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023662758Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023702599Z namespaces/openshift-ingress/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023751571Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023810722Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023862024Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023915035Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.023966537Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024039779Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024274666Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024474571Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024519913Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024567544Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024627466Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024667957Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024746549Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024809491Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024907024Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.024992496Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.025040558Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.025093169Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.025498361Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.025597854Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.02581753Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.025913763Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026155159Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026280223Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026488329Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026572321Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026604082Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026698675Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026727226Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026784107Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026846609Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026934622Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.026955512Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027005574Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027051945Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027082026Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027145428Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027156808Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027167868Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/istiod-openshift-gateway-94bb8fbfd-fwgmd.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027300642Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027315012Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027321013Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.027331863Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.04120453Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.041270642Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.041295912Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.041349054Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.041505889Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.041525399Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.041531429Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.04154459Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.042358223Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.042755184Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.042774715Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router-default-5d495bcbc5-w68bb.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.042919249Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.042938399Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.04295061Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.04296572Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/current.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.043085834Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/previous.insecure.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.043146976Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/previous.log [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.043190427Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.043207587Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.043273129Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.04330517Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.048056836Z [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.048074437Z sent 3,470 bytes received 1,285,723 bytes 859,462.00 bytes/sec [must-gather] [must-gather-8xrfv] OUT 2026-07-08T17:54:54.048088077Z total size is 15,925,641 speedup is 12.35 [must-gather] [must-gather ] OUT 2026-07-08T17:54:54.318021939Z namespace/openshift-must-gather-5rg2q deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: d1dd22ee-2e29-42b5-ba81-0a2594b31f7b [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.23" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-08T17:54:54.447342819Z Using must-gather plug-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ba272a129f91bab6ed70a10b041d49836b63538fa2cbaa919e53a4a13fe3dcde [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: d1dd22ee-2e29-42b5-ba81-0a2594b31f7b [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.23" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-08T17:54:54.467258298Z namespace/openshift-must-gather-h4rcg created [must-gather] [must-gather ] OUT 2026-07-08T17:54:54.475084892Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-7wrq5 created [must-gather] [must-gather ] OUT 2026-07-08T17:54:54.502683872Z pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ba272a129f91bab6ed70a10b041d49836b63538fa2cbaa919e53a4a13fe3dcde created [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:55.730529963Z [disk usage checker] Started [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:55.733585986Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:55.929198674Z Gathering data for ns/openshift-cluster-version... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:56.075447537Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:56.187392844Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:56.280536871Z Gathering data for ns/default... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:56.647025069Z Gathering data for ns/openshift... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:57.089911916Z Gathering data for ns/kube-system... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:57.825564817Z Gathering data for ns/openshift-etcd... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.200783037Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.200809295Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.200809295Z namespaces "assisted-installer" not found [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.475727494Z Waiting on subprocesses to finish execution. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.485809191Z Executing Istio gather script [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.497821256Z INFO: Gathering machine config daemon's old logs from all nodes [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.508343982Z INFO: Gathering on-disk MachineConfig from degraded nodes [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.519816633Z INFO: Collecting host service logs for crio [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.520159586Z INFO: Collecting host service logs for kubelet [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.520537298Z INFO: Collecting host service logs for rpm-ostreed [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.520898077Z INFO: Collecting host service logs for ostree-finalize-staged [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.521249147Z INFO: Collecting host service logs for machine-config-daemon-firstboot [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.521590883Z INFO: Collecting host service logs for machine-config-daemon-host [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.521930229Z INFO: Collecting host service logs for NetworkManager [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.522262177Z INFO: Collecting host service logs for openvswitch [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.522587286Z INFO: Collecting host service logs for ovs-configuration [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.522936776Z INFO: Collecting host service logs for ovsdb-server [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.523290895Z INFO: Collecting host service logs for ovs-vswitchd [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.523685762Z INFO: Waiting for worker host service log collection to complete ... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.526184711Z INFO: Gathering HAProxy config files [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.554344218Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.689997995Z INFO: Waiting for node performance related collection to complete ... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:58.809289386Z INFO: namespace openshift-frr-k8s not detected. Skipping. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.099957856Z INFO: "sriov-network-operator" not detected. Skipping. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.103053496Z INFO: Waiting for on-disk MachineConfig collection to complete ... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.103079876Z INFO: on-disk MachineConfig config collection complete. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.203813559Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-n9nrj [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.203856765Z insights-runtime-extractor-92n4w [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.203873766Z insights-runtime-extractor-dtzt4 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.203890522Z insights-runtime-extractor-f2vbk [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.476177448Z No resources found in openshift-etcd namespace. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.521670660Z INFO: "metallb-operator" not detected. Skipping. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.653479766Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.852458904Z error: the server doesn't have a resource type "performanceprofile" [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:54:59.876596321Z ERROR: No running kube-apiserver pods found [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.026594530Z INFO: Found 1 replicas - prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.040532142Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.040532142Z See 'oc rsync -h' for help and examples [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.215762497Z No resources found [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.259298451Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.367820474Z INFO: Worker host service log collection to complete. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.632458486Z error: the server doesn't have a resource type "multi-networkpolicy" [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.638994612Z No resources found [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:00.803680674Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.117121258Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.134542343Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.155169586Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.300898426Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.337070241Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.337202513Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.371235854Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istio-reader-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.405527668Z INFO: Getting alertmanagers from prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.449080575Z INFO: Waiting for HAProxy config collection to complete ... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.671064764Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.698152382Z INFO: 'previous-logs' folder not found on ip-10-0-129-204.ec2.internal, skipping... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.715169135Z error: the server doesn't have a resource type "machineconfigs" [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.815743283Z tar: Removing leading `/' from member names [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.822561938Z INFO: HAProxy config collection complete. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:01.871865278Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.050246930Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.054267007Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istiod-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.101818180Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.118306998Z error: the server doesn't have a resource type "machineconfigpools" [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.242518638Z INFO: Getting rules from prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.254577149Z INFO: 'previous-logs' folder not found on ip-10-0-140-244.ec2.internal, skipping... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.450345036Z W0708 17:55:02.450304 1105 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.476738801Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.504052246Z W0708 17:55:02.504004 1148 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.537708149Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.548448353Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.556749605Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istiod-gateway-controller-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.568223741Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.826543134Z INFO: 'previous-logs' folder not found on ip-10-0-142-38.ec2.internal, skipping... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.826663625Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.826715006Z INFO: Machine Config Daemon termination log collection complete. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.830730798Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.875231506Z error: the server doesn't have a resource type "kubeletconfigs" [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:02.903680077Z Gathering data for ns/openshift-monitoring... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.031927544Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.033427139Z INFO: Getting status/config from prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.035689822Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istio-reader-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.079787758Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.259884621Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.374555852Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.379290929Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istiod-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.519500541Z INFO: INTERCONNECT MODE [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.519530634Z INFO: Gathering ovn-kubernetes DBs [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.559142098Z INFO: Getting status/flags from prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.645578576Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.690319483Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.693657825Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istiod-gateway-controller-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.739606708Z INFO: Gathering OVN_Northbound from ovnkube-node-22jbf... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.739860631Z INFO: Gathering OVN_Northbound from ovnkube-node-ndw9q... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.740114604Z INFO: Gathering OVN_Northbound from ovnkube-node-sntx9... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.740383932Z INFO: Gathering OVN_Southbound from ovnkube-node-22jbf... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.740624411Z INFO: Gathering OVN_Southbound from ovnkube-node-ndw9q... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.741116473Z INFO: Gathering OVN_Southbound from ovnkube-node-sntx9... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:03.914617650Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.085983343Z tar: Removing leading `/' from member names [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.099230805Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.166558328Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.168302509Z tar: Removing leading `/' from member names [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.170243644Z tar: Removing leading `/' from member names [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.273911933Z tar: Removing leading `/' from member names [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.305730407Z tar: Removing leading `/' from member names [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.305852900Z tar: Removing leading `/' from member names [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.374905201Z INFO: Getting targets?state=active from prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.456775674Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.705670602Z INFO: Getting status/tsdb from prometheus-k8s-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.791187198Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.828853554Z INFO: Waiting for network log collection to complete ... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.828897241Z INFO: Waiting for ovnk database copies to complete ... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.829391828Z INFO: Copying ovnk databases complete. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.831558663Z 7.2M must-gather/network_logs/ovnk_database_store [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.834919306Z ovnk_database_store/ [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.834977010Z ovnk_database_store/ovnkube-node-22jbf_nbdb [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.852606714Z ovnk_database_store/ovnkube-node-22jbf_sbdb [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.898009878Z ovnk_database_store/ovnkube-node-ndw9q_nbdb [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.915511038Z ovnk_database_store/ovnkube-node-ndw9q_sbdb [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:04.959863360Z ovnk_database_store/ovnkube-node-sntx9_sbdb [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.007206549Z ovnk_database_store/ovnkube-node-sntx9_nbdb [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.028255843Z INFO: Network log collection complete. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.152081132Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.195434797Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.206552560Z INFO: Getting status from alertmanager-main-0 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.502815889Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.571506248Z Gathering data for ns/openshift-network-console... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.774051352Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.776255539Z INFO: Fallback to identify the container image from release info [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:05.818493680Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.077768979Z Gathering data for ns/openshift-console-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.258202856Z Inspecting resource crd/authorizationpolicies.security.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.346429039Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.463150425Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.492488764Z daemonset.apps/perf-node-gather-daemonset created [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.546292886Z Gathering data for ns/openshift-console... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.643131310Z Waiting for performance profile collector pods to become ready: 1 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.719947077Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.722145258Z Inspecting resource crd/destinationrules.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:06.963743330Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.196714659Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.200166258Z Inspecting resource crd/envoyfilters.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.232997056Z Gathering data for ns/openshift-cluster-storage-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.389844734Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.602809879Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.605866380Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.792535502Z Waiting for performance profile collector pods to become ready: 2 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.811681623Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:07.871205095Z Gathering data for ns/openshift-dns-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.023161655Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.025927201Z Inspecting resource crd/gateways.gateway.networking.k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.230009380Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.234790406Z Gathering data for ns/openshift-dns... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.412563950Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.414888673Z Inspecting resource crd/gateways.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.595295552Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.812079042Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.814647446Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.936224246Z Gathering data for ns/openshift-image-registry... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:08.956020979Z Waiting for performance profile collector pods to become ready: 3 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.043311954Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.249551078Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.251947164Z Inspecting resource crd/httproutes.gateway.networking.k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.478605853Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.544209665Z Gathering data for ns/openshift-ingress-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.694858594Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.697058808Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.892742522Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:09.950344097Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.110686760Z Waiting for performance profile collector pods to become ready: 4 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.130296030Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.133506726Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.312933257Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.528919180Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.531069467Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.699829940Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.830584513Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.859716041Z Gathering data for ns/openshift-ingress-canary... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.907859571Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:10.910033204Z Inspecting resource crd/inferencepools.inference.networking.k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.090699068Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.255285191Z Waiting for performance profile collector pods to become ready: 5 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.322688070Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.326005963Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.364708470Z Gathering data for ns/openshift-insights... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.490700086Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.701239741Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.703445413Z Inspecting resource crd/peerauthentications.security.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:11.882981210Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.087516231Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.090871751Z Inspecting resource crd/proxyconfigs.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.260703785Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.388588753Z Waiting for performance profile collector pods to become ready: 6 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.463426500Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.465821635Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.623847490Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.829270702Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:12.831784704Z Inspecting resource crd/requestauthentications.security.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.004878205Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.207205126Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.217807835Z Inspecting resource crd/serviceentries.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.384598566Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.536297625Z Waiting for performance profile collector pods to become ready: 7 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.617407506Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.620541081Z Inspecting resource crd/sidecars.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:13.802715577Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.014732711Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.018232941Z Inspecting resource crd/telemetries.telemetry.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.193412867Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.196927616Z Gathering data for ns/openshift-lws-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.393796137Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.396022526Z Inspecting resource crd/virtualservices.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.612596069Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.683790137Z Daemonset perf-node-gather-daemonset ready 3 out of 3 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.825155961Z Gathering data for ns/kserve... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.833932738Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.836416023Z Inspecting resource crd/wasmplugins.extensions.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:14.991968802Z Collecting performance related data for node ip-10-0-140-244.ec2.internal [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.002398735Z Collecting performance related data for node ip-10-0-142-38.ec2.internal [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.012756272Z Collecting performance related data for node ip-10-0-129-204.ec2.internal [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.060697347Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.323902828Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.327593854Z Inspecting resource crd/workloadentries.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.536669956Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.770796640Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.774238981Z Inspecting resource crd/workloadgroups.networking.istio.io [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:15.837126612Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:16.032144938Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:16.300586528Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:16.443667174Z Inspecting resource mutatingwebhookconfiguration.admissionregistration.k8s.io/istio-sidecar-injector-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:16.720969823Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:17.236634158Z Gathering data for ns/openshift-config... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:17.353059766Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:17.524208939Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:17.714405018Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:17.808728267Z Gathering data for ns/openshift-config-managed... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:17.878056565Z Inspecting resource validatingwebhookconfiguration.admissionregistration.k8s.io/istio-validator-openshift-gateway-openshift-ingress [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:18.114416017Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:18.411048986Z Gathering data for ns/openshift-kube-apiserver-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:18.740355139Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:18.793346602Z Gathering data for ns/openshift-kube-apiserver... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:18.948694701Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:19.099981287Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:19.195881963Z Gathering data for ns/openshift-kube-controller-manager... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:19.573060992Z Done executing Istio gather script [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:19.586370127Z Gathering data for ns/openshift-kube-controller-manager-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:19.935237638Z Gathering data for ns/openshift-kube-scheduler... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:20.239915220Z Gathering data for ns/openshift-kube-scheduler-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:20.631148243Z Gathering data for ns/openshift-kube-storage-version-migrator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:20.843758373Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:21.009331219Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:21.376280904Z Gathering data for ns/openshift-user-workload-monitoring... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:21.958352774Z Gathering data for ns/openshift-multus... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:23.463020785Z Gathering data for ns/openshift-ovn-kubernetes... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:23.544597119Z Collecting kubelet logs for node ip-10-0-140-244.ec2.internal [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:23.546269171Z Collecting kubelet logs for node ip-10-0-142-38.ec2.internal [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:23.547993137Z Collecting kubelet logs for node ip-10-0-129-204.ec2.internal [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:23.958440870Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-h4rcg namespace [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:23.963388621Z INFO: Node performance data collection complete. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:24.910176621Z Gathering data for ns/openshift-host-network... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:25.252868433Z Gathering data for ns/openshift-network-diagnostics... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:25.814144918Z Gathering data for ns/openshift-network-node-identity... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:25.850410806Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:26.303895064Z Gathering data for ns/openshift-network-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:26.720469617Z Gathering data for ns/openshift-cloud-network-config-controller... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:27.067586946Z Gathering data for ns/openshift-cluster-node-tuning-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:27.588702507Z Gathering data for ns/openshift-apiserver-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:27.934177033Z Gathering data for ns/openshift-apiserver... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:28.387727353Z Gathering data for ns/openshift-controller-manager-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:28.734697819Z Gathering data for ns/openshift-controller-manager... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:29.070760345Z Gathering data for ns/openshift-cluster-samples-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:29.758129509Z Gathering data for ns/openshift-operator-lifecycle-manager... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:30.146554501Z Gathering data for ns/openshift-service-ca-operator... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:30.496799594Z Gathering data for ns/openshift-service-ca... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:30.856438695Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70 [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:30.920364226Z Gathering data for ns/openshift-cluster-csi-drivers... [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:31.869070749Z Wrote inspect data to must-gather. [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:31.869145659Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:31.869145659Z skipping gathering secrets/support due to error: secrets "support" not found [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:31.997646572Z error: the server doesn't have a resource type "clusters" [must-gather] [must-gather-2sn4t] POD 2026-07-08T17:55:32.184202060Z Caches written to disk [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.197980494Z waiting for gather to complete [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.201896596Z downloading gather output [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.468154181Z receiving incremental file list [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.48937949Z ./ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.489478323Z aggregated-discovery-api.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.489763951Z aggregated-discovery-apis.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.490681707Z event-filter.html [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.493369364Z timestamp [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.493465067Z version [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500250892Z cluster-scoped-resources/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500315123Z cluster-scoped-resources/admissionregistration.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500329884Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500459898Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/default-network-annotation.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500623763Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/servicecidrs.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500742276Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/user-defined-networks-namespace-label.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500854439Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.500904391Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/default-network-annotation-binding.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.501034234Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/servicecidrs-binding.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.501137087Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/user-defined-networks-namespace-label-binding.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.50125733Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.501293122Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/multus.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.501417595Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/network-node-identity.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.501507058Z cluster-scoped-resources/apiextensions.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.50195181Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.502005462Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/accounts.nim.opendatahub.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.502194527Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.502444844Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminpolicybasedexternalroutes.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.502660921Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertingrules.monitoring.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.502811935Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagerconfigs.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.506372627Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagers.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.507941182Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertrelabelconfigs.monitoring.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.508052705Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeyapprovals.devportal.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.508159868Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeyrequests.devportal.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.508278092Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeys.devportal.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.508399705Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiproducts.devportal.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.50890352Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apirequestcounts.apiserver.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.509034073Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiservers.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.509186108Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/appliedmanifestworks.work.open-cluster-management.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.509299221Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authconfigs.authorino.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.510069973Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authentications.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.51030996Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorinos.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.510440074Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorizationpolicies.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.510591478Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authpolicies.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.511816864Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/baselineadminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.512045Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/builds.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.512205815Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/bundles.trust.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.512359469Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/catalogsources.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.512669578Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificaterequests.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.512843383Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificates.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.513039299Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certmanagers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.513289906Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/challenges.acme.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.513952285Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudcredentials.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.514090329Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.514196642Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudprivateipconfigs.cloud.network.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.514287074Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterclaims.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.514391477Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.51450365Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercsidrivers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.514694856Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterimagepolicies.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.514894192Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterissuers.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.515631893Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteroperators.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.515754006Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterresourcequotas.quota.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.51589297Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterserviceversions.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.517813735Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterstoragecontainers.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.51797183Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustertriggerauthentications.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.518121374Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteruserdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.518349121Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterversions.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.518641869Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.519121313Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.519236426Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.samples.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.51937864Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleclidownloads.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.519474293Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleexternalloglinks.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.519582076Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolelinks.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.519701039Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolenotifications.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.519818223Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleplugins.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.519997238Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolequickstarts.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.520134392Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.520242375Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.520498862Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolesamples.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.520646517Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleyamlsamples.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.52075661Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/controllerconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.521313096Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/credentialsrequests.cloudcredential.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.521422719Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/csisnapshotcontrollers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.521543372Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/datascienceclusters.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.521728148Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/destinationrules.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.522470529Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.522583602Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.522763017Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnshealthcheckprobes.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.522904621Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnspolicies.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.523076126Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.ingress.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.52320802Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.523366964Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dscinitializations.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.523497648Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressfirewalls.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.523611391Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressips.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.523729905Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressqoses.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.523872039Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressrouters.network.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.524000493Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressservices.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.524122116Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/envoyfilters.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.52425966Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/etcds.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.524392534Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/featuregates.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.524553659Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.524694283Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.525071664Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.525243569Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/grpcroutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.525602479Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/helmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.525716392Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httproutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.526581197Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httpscaledobjects.http.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.526720261Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentpolicies.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.526870795Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentsourcepolicies.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.526981708Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagedigestmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.527097071Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepolicies.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.527284757Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepruners.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.527528444Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/images.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.527656098Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagetagmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.527777391Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencegraphs.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.527953696Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencemodelrewrites.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.52808017Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceobjectives.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.528198273Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepoolimports.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.528333017Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.528467731Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.528596374Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.530698435Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/infrastructures.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.531147978Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresscontrollers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.531739065Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresses.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.53193031Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/insightsoperators.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.532096355Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/installplans.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.532233879Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/interceptorroutes.http.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.532368103Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ipamclaims.k8s.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.532478716Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ippools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.532583179Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/issuers.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.533356491Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istiocsrs.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.53365957Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kedacontrollers.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.536502321Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kuadrants.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.536631495Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeapiservers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.539280251Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubecontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.539716393Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeschedulers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.539983081Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubestorageversionmigrators.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.540133945Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersetoperators.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.540251669Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersets.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.543229534Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/limitadors.limitador.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.543545153Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceserviceconfigs.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.548875676Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.553002634Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfignodes.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.553151089Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfigurations.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.553430647Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosbuilds.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.553592441Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.553739395Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/network-attachment-definitions.k8s.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.553868079Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.554017643Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.5542505Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodes.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.554369084Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodeslicepools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.554476977Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oauths.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.554663962Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oidcpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.554811836Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/olmconfigs.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.55493477Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftapiservers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.555043583Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftcontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.555157526Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorconditions.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.55529659Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorgroups.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.555444244Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorhubs.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.555578108Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorpkis.network.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.555700732Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operators.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.555871807Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/orders.acme.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.556041172Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/overlappingrangeipreservations.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.556137144Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/peerauthentications.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.556273188Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/pinnedimagesets.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.556378861Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/planpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.556500485Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podmonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.556762742Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podnetworkconnectivitychecks.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.556928427Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/probes.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.557163284Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/profiles.tuned.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.557281777Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projecthelmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.55738927Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projects.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.557493733Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheuses.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.55948335Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheusrules.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.559615454Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxies.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.559725527Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxyconfigs.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.559849531Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rangeallocations.security.internal.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.559957384Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.560099648Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.560222372Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/requestauthentications.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.560371096Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rolebindingrestrictions.authorization.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.560490959Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledjobs.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.561994242Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledobjects.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.562140556Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/schedulers.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.56225602Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/securitycontextconstraints.security.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.562414124Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicecas.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.562536038Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serviceentries.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.562721793Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicemonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.562992571Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servingruntimes.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.563504686Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/sidecars.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.563767463Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storages.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.563907987Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storagestates.migration.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.56401937Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storageversionmigrations.migration.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.564121153Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/subscriptions.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.56471228Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetries.telemetry.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.564925136Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetrypolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.56505415Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/thanosrulers.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.56642995Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tlspolicies.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.566576084Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tokenratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.566726028Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trainedmodels.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.566850102Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/triggerauthentications.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.567003096Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trustmanagers.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.567297544Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tuneds.tuned.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.567437278Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/userdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.567607633Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/virtualservices.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.568088467Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumepopulators.populator.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.56819352Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotclasses.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.568304903Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotcontents.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.568489158Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshots.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.568633823Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/wasmplugins.extensions.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.568763636Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadentries.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.568932401Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadgroups.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.569067405Z cluster-scoped-resources/apiregistration.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.569373504Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.569424885Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1..yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.569513528Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.acme.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.5695911Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.569670192Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiextensions.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.569748565Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiserver.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.569921019Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570024462Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570106915Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authentication.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570201398Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57028014Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570371703Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.autoscaling.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570457045Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.batch.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570544117Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.build.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57063859Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570723573Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.certificates.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570833146Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloud.network.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.570935129Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloudcredential.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571019051Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.config.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571105184Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.console.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571216497Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.coordination.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57133742Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571441963Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.discovery.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571525616Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571618208Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.events.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57170171Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.flowcontrol.apiserver.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571808064Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.571930207Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.image.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57201882Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572110642Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.inference.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572196875Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.ingress.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572281457Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57237165Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.ovn.org.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572460623Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572549865Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572640648Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572740951Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572853964Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.572942876Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.network.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573030809Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573121061Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573203574Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.nim.opendatahub.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573293276Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.node.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573380849Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.oauth.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573469151Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573553414Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573641906Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.packages.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573736409Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.policy.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573867613Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.project.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.573963186Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.quota.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574050218Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.rbac.authorization.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57413786Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.resource.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574227683Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.route.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574317206Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.samples.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574404518Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.scheduling.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574494041Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.internal.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574588324Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574687976Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574772459Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574888692Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.574975775Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.telemetry.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575072747Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.template.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57515892Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.tuned.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575276483Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.user.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575389806Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.work.open-cluster-management.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57551851Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575612443Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575706505Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.devportal.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575807408Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.eventing.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575906311Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.575995034Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576082116Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.http.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576168079Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576263752Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576347904Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576437337Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576530599Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.limitador.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576621562Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.migration.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576722715Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.576836968Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operator.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57692344Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577017683Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.policy.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577106756Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577195738Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.telemetry.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577285071Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.trust.cert-manager.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577374443Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577461516Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577549908Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577642821Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577733554Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha3.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577841477Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.577932709Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.external.metrics.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578043132Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578148915Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.helm.openshift.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578246438Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.http.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578361622Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578449904Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.metrics.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578543697Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.monitoring.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578634909Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578732892Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578833835Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.populator.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.578934148Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579024631Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.serving.kserve.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579213956Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.storage.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579316609Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta3.authorino.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579399882Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.autoscaling.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579489514Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.operators.coreos.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579543375Z cluster-scoped-resources/certificates.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579550556Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579595647Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/csr-fx4xw.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.57971068Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-gbzq8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.579898776Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-l4dq4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58006343Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-ns844.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.580202014Z cluster-scoped-resources/config.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.580251746Z cluster-scoped-resources/config.openshift.io/apiservers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.580420141Z cluster-scoped-resources/config.openshift.io/authentications.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.580573555Z cluster-scoped-resources/config.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.580677498Z cluster-scoped-resources/config.openshift.io/clusterimagepolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.580767921Z cluster-scoped-resources/config.openshift.io/clusteroperators.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581107871Z cluster-scoped-resources/config.openshift.io/clusterversions.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581217034Z cluster-scoped-resources/config.openshift.io/consoles.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581297256Z cluster-scoped-resources/config.openshift.io/dnses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581388279Z cluster-scoped-resources/config.openshift.io/featuregates.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581495782Z cluster-scoped-resources/config.openshift.io/imagecontentpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581577424Z cluster-scoped-resources/config.openshift.io/imagedigestmirrorsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581663666Z cluster-scoped-resources/config.openshift.io/imagepolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581750819Z cluster-scoped-resources/config.openshift.io/images.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581862152Z cluster-scoped-resources/config.openshift.io/imagetagmirrorsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.581944834Z cluster-scoped-resources/config.openshift.io/infrastructures.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582049897Z cluster-scoped-resources/config.openshift.io/ingresses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58213852Z cluster-scoped-resources/config.openshift.io/networks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582239293Z cluster-scoped-resources/config.openshift.io/nodes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582325115Z cluster-scoped-resources/config.openshift.io/oauths.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582412658Z cluster-scoped-resources/config.openshift.io/operatorhubs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58249704Z cluster-scoped-resources/config.openshift.io/projects.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582603393Z cluster-scoped-resources/config.openshift.io/proxies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582684505Z cluster-scoped-resources/config.openshift.io/schedulers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582777788Z cluster-scoped-resources/config.openshift.io/clusteroperators/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58284262Z cluster-scoped-resources/config.openshift.io/clusteroperators/console.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.582950133Z cluster-scoped-resources/config.openshift.io/clusteroperators/csi-snapshot-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583049656Z cluster-scoped-resources/config.openshift.io/clusteroperators/dns.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583149119Z cluster-scoped-resources/config.openshift.io/clusteroperators/image-registry.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583252822Z cluster-scoped-resources/config.openshift.io/clusteroperators/ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583363795Z cluster-scoped-resources/config.openshift.io/clusteroperators/insights.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583467948Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583610572Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583697325Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-scheduler.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583784027Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-storage-version-migrator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.583902971Z cluster-scoped-resources/config.openshift.io/clusteroperators/monitoring.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584009054Z cluster-scoped-resources/config.openshift.io/clusteroperators/network.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584136647Z cluster-scoped-resources/config.openshift.io/clusteroperators/node-tuning.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.5842241Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584312972Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584406515Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-samples.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584497377Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-catalog.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58458022Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-packageserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584668733Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584762565Z cluster-scoped-resources/config.openshift.io/clusteroperators/service-ca.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584874838Z cluster-scoped-resources/config.openshift.io/clusteroperators/storage.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58494168Z cluster-scoped-resources/config.openshift.io/clusterversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.584992612Z cluster-scoped-resources/config.openshift.io/clusterversions/version.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585072104Z cluster-scoped-resources/config.openshift.io/consoles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585113595Z cluster-scoped-resources/config.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585177757Z cluster-scoped-resources/config.openshift.io/featuregates/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585209498Z cluster-scoped-resources/config.openshift.io/featuregates/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58529378Z cluster-scoped-resources/config.openshift.io/infrastructures/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585313681Z cluster-scoped-resources/config.openshift.io/infrastructures/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585389913Z cluster-scoped-resources/config.openshift.io/oauths/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585405124Z cluster-scoped-resources/config.openshift.io/oauths/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585485746Z cluster-scoped-resources/config.openshift.io/proxies/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585518197Z cluster-scoped-resources/config.openshift.io/proxies/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585603509Z cluster-scoped-resources/console.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58561226Z cluster-scoped-resources/console.openshift.io/consoleplugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585671631Z cluster-scoped-resources/console.openshift.io/consoleplugins/kuadrant-console-plugin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585771104Z cluster-scoped-resources/console.openshift.io/consoleplugins/monitoring-plugin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585878387Z cluster-scoped-resources/console.openshift.io/consoleplugins/networking-console-plugin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585928799Z cluster-scoped-resources/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.585939359Z cluster-scoped-resources/core/nodes/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58598934Z cluster-scoped-resources/core/nodes/ip-10-0-129-204.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586174556Z cluster-scoped-resources/core/nodes/ip-10-0-140-244.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586304019Z cluster-scoped-resources/core/nodes/ip-10-0-142-38.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586410892Z cluster-scoped-resources/core/persistentvolumes/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586460894Z cluster-scoped-resources/core/persistentvolumes/pvc-42130d34-0182-4d20-95af-c345cf3e144c.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586536466Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586595118Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586639459Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/catch-all.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586747462Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/exempt.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586851315Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/global-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.586929507Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58702246Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-scheduler.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587136303Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-system-service-accounts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587311978Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587433872Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-sar.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587554425Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587647308Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-authentication-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587751791Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587868554Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.587960857Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-monitoring-metrics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588044979Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver-sar.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588142892Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588229045Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-server.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588317487Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/probes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.58840752Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/service-accounts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588499632Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-leader-election.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588586965Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-node-high.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588676507Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-nodes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.5887575Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588809981Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/catch-all.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.588995897Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/exempt.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589071889Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/global-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589157891Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/leader-election.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589236804Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/node-high.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589318206Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/openshift-control-plane-operators.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.5894663Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/system.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589587803Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-high.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589673436Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-low.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589742558Z cluster-scoped-resources/gateway.networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589749878Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.5898036Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/openshift-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589913403Z cluster-scoped-resources/imageregistry.operator.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589929423Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.589962634Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590200251Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590232752Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590295964Z cluster-scoped-resources/migration.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590305664Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590364576Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/console-plugin-storage-version-migration.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590452668Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-controllerconfig-storage-version-migration.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590599383Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-machineconfigpool-storage-version-migration.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590649894Z cluster-scoped-resources/oauth.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590657124Z cluster-scoped-resources/oauth.openshift.io/oauthclients/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590700806Z cluster-scoped-resources/oauth.openshift.io/oauthclients/console.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590764047Z cluster-scoped-resources/operator.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590771488Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590830199Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590912712Z cluster-scoped-resources/operator.openshift.io/consoles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.590942182Z cluster-scoped-resources/operator.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591049665Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591079336Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591161429Z cluster-scoped-resources/operator.openshift.io/dnses/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591191749Z cluster-scoped-resources/operator.openshift.io/dnses/default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591269632Z cluster-scoped-resources/operator.openshift.io/insightsoperators/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591315133Z cluster-scoped-resources/operator.openshift.io/insightsoperators/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591429726Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591459497Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591528899Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591577881Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591631142Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591658723Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591743016Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591778766Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59189162Z cluster-scoped-resources/operator.openshift.io/networks/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.591940281Z cluster-scoped-resources/operator.openshift.io/networks/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592034224Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592091215Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592139167Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592187678Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59223187Z cluster-scoped-resources/operator.openshift.io/servicecas/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592273891Z cluster-scoped-resources/operator.openshift.io/servicecas/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592309492Z cluster-scoped-resources/operator.openshift.io/storages/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592350753Z cluster-scoped-resources/operator.openshift.io/storages/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592451396Z cluster-scoped-resources/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592458456Z cluster-scoped-resources/operators.coreos.com/olmconfigs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592498197Z cluster-scoped-resources/operators.coreos.com/olmconfigs/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592574129Z cluster-scoped-resources/operators.coreos.com/operators/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59261513Z cluster-scoped-resources/operators.coreos.com/operators/authorino-operator.kuadrant-system.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592736414Z cluster-scoped-resources/operators.coreos.com/operators/dns-operator.kuadrant-system.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.592873978Z cluster-scoped-resources/operators.coreos.com/operators/leader-worker-set.openshift-lws-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593034982Z cluster-scoped-resources/operators.coreos.com/operators/limitador-operator.kuadrant-system.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593135785Z cluster-scoped-resources/operators.coreos.com/operators/openshift-cert-manager-operator.cert-manager-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593247059Z cluster-scoped-resources/operators.coreos.com/operators/openshift-custom-metrics-autoscaler-operator.openshift-keda.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593348002Z cluster-scoped-resources/operators.coreos.com/operators/rhcl-operator.kuadrant-system.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593438104Z cluster-scoped-resources/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593499146Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593550847Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/aws-ebs-csi-driver-operator-clusterrolebinding.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593632199Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cloud-network-config-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593761113Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cluster-node-tuning-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.593894857Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cluster-storage-operator-role.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594021661Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/csi-snapshot-controller-operator-clusterrole.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594111193Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/csi-snapshot-controller-runner-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594213336Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/metrics-daemon-sa-rolebinding.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594309359Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-admission-controller-webhook.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594415522Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-ancillary-tools.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594515135Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-cluster-readers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594601777Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-group.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59469345Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-transient.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594806363Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-whereabouts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594899206Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-diagnostics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.594981868Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-node-identity.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595067671Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-image-registry-pruner.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595162203Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-iptables-alerter.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595257216Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595368679Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-identity-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595459232Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-kube-rbac-proxy.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595549594Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/registry-registry-role.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595662498Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595702909Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/aws-ebs-csi-driver-operator-clusterrole.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595806962Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/cloud-network-config-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595912865Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/cluster-node-tuning-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.595999947Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console-extensions-reader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.596140782Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.596234904Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.596322917Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/csi-snapshot-controller-operator-clusterrole.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.596414089Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/helm-chartrepos-viewer.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.596500532Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/metrics-daemon-role.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.596587994Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-admission-controller-webhook.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.596693297Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-ancillary-tools.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59678529Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597005866Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/net-attach-def-project.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597099209Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-diagnostics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597192182Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-node-identity.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597284234Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-csi-snapshot-controller-runner.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597373647Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-iptables-alerter.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597456809Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-cluster-reader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597542582Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597635304Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-kube-rbac-proxy.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597720007Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59783389Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-editor.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.597922413Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-viewer.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598010845Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/project-helm-chartrepository-editor.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598099308Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-admin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59818855Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-basic-user.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598281283Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-storage-admin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598366315Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-view.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598454328Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:registry.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598548591Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/whereabouts-cni.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598597612Z cluster-scoped-resources/samples.operator.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598604592Z cluster-scoped-resources/samples.operator.openshift.io/configs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598647943Z cluster-scoped-resources/samples.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598698105Z cluster-scoped-resources/snapshot.storage.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598705185Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598758666Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/csi-aws-vsc.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598832749Z cluster-scoped-resources/storage.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598845649Z cluster-scoped-resources/storage.k8s.io/csidrivers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59889028Z cluster-scoped-resources/storage.k8s.io/csidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598975183Z cluster-scoped-resources/storage.k8s.io/csinodes/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.598995633Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-129-204.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599119937Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-140-244.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59921617Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-142-38.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599285082Z cluster-scoped-resources/storage.k8s.io/storageclasses/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599346613Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp2-csi.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599443116Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp3-csi.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599515568Z host_service_logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599531079Z host_service_logs/masters/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59956624Z host_service_logs/masters/NetworkManager_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599635482Z host_service_logs/masters/crio_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599688273Z host_service_logs/masters/kubelet_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.599761465Z host_service_logs/masters/machine-config-daemon-firstboot_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.59991571Z host_service_logs/masters/machine-config-daemon-host_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600021353Z host_service_logs/masters/openvswitch_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600091515Z host_service_logs/masters/ostree-finalize-staged_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600165877Z host_service_logs/masters/ovs-configuration_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600231409Z host_service_logs/masters/ovs-vswitchd_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600300061Z host_service_logs/masters/ovsdb-server_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600375083Z host_service_logs/masters/rpm-ostreed_service.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600418724Z ingress_controllers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600428045Z ingress_controllers/default/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600432754Z ingress_controllers/default/router-default-5d495bcbc5-w68bb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600480326Z ingress_controllers/default/router-default-5d495bcbc5-w68bb/haproxy.config [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.60060277Z insights-data/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.60061264Z istio/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600673361Z istio/aggregated-discovery-api.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.600782455Z istio/aggregated-discovery-apis.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601467304Z istio/event-filter.html [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601625599Z istio/timestamp [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601763093Z istio/cluster-scoped-resources/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601772253Z istio/cluster-scoped-resources/admissionregistration.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601780873Z istio/cluster-scoped-resources/admissionregistration.k8s.io/mutatingwebhookconfigurations/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601846735Z istio/cluster-scoped-resources/admissionregistration.k8s.io/mutatingwebhookconfigurations/istio-sidecar-injector-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601927457Z istio/cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.601972979Z istio/cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/istio-validator-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.60203157Z istio/cluster-scoped-resources/apiextensions.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.602111623Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.602166814Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorizationpolicies.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.602338459Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/destinationrules.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.603132362Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/envoyfilters.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.603388449Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.603559224Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.604071529Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.604244204Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/grpcroutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.604648315Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httproutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.605432178Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencemodelrewrites.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.605556581Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceobjectives.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.605708566Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepoolimports.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.605988044Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.606150038Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.606287593Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/peerauthentications.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.606412396Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxyconfigs.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.606522759Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.606638532Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/requestauthentications.security.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.606785867Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serviceentries.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.606984122Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/sidecars.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.60723458Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetries.telemetry.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.607427715Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/virtualservices.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.607813166Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/wasmplugins.extensions.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.60796195Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadentries.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608079534Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadgroups.networking.istio.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608223528Z istio/cluster-scoped-resources/gateway.networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608232648Z istio/cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.60828456Z istio/cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/openshift-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608381862Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608391503Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608424944Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istio-reader-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608536757Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istiod-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608697111Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istiod-gateway-controller-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608761693Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608820235Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istio-reader-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.608928278Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istiod-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609096903Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istiod-gateway-controller-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609167505Z istio/namespaces/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609184985Z istio/namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609193096Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609207326Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609249017Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.60935417Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-2.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609441993Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609489744Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609588427Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-2.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609663139Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-3.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609753972Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/router-route-4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609858425Z istio/namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/httproutes/scheduler-v06-pd-migration-test-kserve-route.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609989679Z istio/namespaces/kserve-ci-e2e-test/inference.networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.609997009Z istio/namespaces/kserve-ci-e2e-test/inference.networking.k8s.io/inferencepools/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.61002745Z istio/namespaces/kserve-ci-e2e-test/inference.networking.k8s.io/inferencepools/scheduler-v06-pd-migration-test-inference-pool.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610088162Z istio/namespaces/kserve-ci-e2e-test/inference.networking.x-k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610095382Z istio/namespaces/kserve-ci-e2e-test/inference.networking.x-k8s.io/inferencepools/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610130343Z istio/namespaces/kserve-ci-e2e-test/inference.networking.x-k8s.io/inferencepools/scheduler-v06-pd-migration-test-inference-pool.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610192714Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610201745Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610256416Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/scheduler-v06-pd-migration-test-kserve-scheduler.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.61037907Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/scheduler-v06-pd-migration-test-kserve-shadow-svc.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610529004Z istio/namespaces/kserve-ci-e2e-test/networking.istio.io/destinationrules/scheduler-v06-pd-migration-test-kserve-workload-svc.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610606436Z istio/namespaces/openshift-ingress/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610656578Z istio/namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610713489Z istio/namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.61075104Z istio/namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610836453Z istio/namespaces/openshift-ingress/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610889044Z istio/namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.610979377Z istio/namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.611417369Z istio/namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.611594405Z istio/namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.611654436Z istio/namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.611698758Z istio/namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.61176714Z istio/namespaces/openshift-ingress/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.611820951Z istio/namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.611920144Z istio/namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.611978956Z istio/namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.612029297Z istio/namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.61211064Z istio/namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.612182091Z istio/namespaces/openshift-ingress/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.612225373Z istio/namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.612638535Z istio/namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.612741098Z istio/namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613021035Z istio/namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613103168Z istio/namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613355045Z istio/namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613499239Z istio/namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613695045Z istio/namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613780227Z istio/namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613849479Z istio/namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613928912Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613938422Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.613961113Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/openshift-ai-inference.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614047945Z istio/namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614088416Z istio/namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614162408Z istio/namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614204949Z istio/namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614289052Z istio/namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614365284Z istio/namespaces/openshift-ingress/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614400015Z istio/namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614457067Z istio/namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614514158Z istio/namespaces/openshift-ingress/networking.istio.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614525569Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614576Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/kuadrant-auth-openshift-ai-inference.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614667713Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/kuadrant-openshift-ai-inference.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.614786926Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/openshift-ai-inference-authn-ssl.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615016343Z istio/namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615059134Z istio/namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615132776Z istio/namespaces/openshift-ingress/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615140276Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615191608Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/istiod-openshift-gateway-94bb8fbfd-fwgmd.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615318032Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615329272Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615335692Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.615381873Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629427886Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629489728Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629552049Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629581201Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629733075Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629753735Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629765666Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.629859818Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63062324Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631237988Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631280629Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router-default-5d495bcbc5-w68bb.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631447444Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631459794Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631471175Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631524356Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.6316384Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631707651Z istio/namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631722312Z istio/namespaces/openshift-ingress/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631773153Z istio/namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631845505Z istio/namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631882756Z istio/namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631933448Z machine_config_ondisk/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631942648Z machine_config_termination_logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631961119Z monitoring/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.631980899Z monitoring/alertmanager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63199654Z monitoring/alertmanager/status.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.632095693Z monitoring/alertmanager/status.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.632160234Z monitoring/prometheus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.632201896Z monitoring/prometheus/alertmanagers.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.632293158Z monitoring/prometheus/alertmanagers.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.6323529Z monitoring/prometheus/rules.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.632979068Z monitoring/prometheus/rules.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.633029359Z monitoring/prometheus/prometheus-k8s-0/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.633075381Z monitoring/prometheus/prometheus-k8s-0/active-targets.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.633903264Z monitoring/prometheus/prometheus-k8s-0/active-targets.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.633963746Z monitoring/prometheus/prometheus-k8s-0/status/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634006377Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.6340946Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634161442Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634255904Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634282665Z monitoring/prometheus/status/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634331006Z monitoring/prometheus/status/config.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634773059Z monitoring/prometheus/status/config.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634862222Z monitoring/prometheus/status/flags.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.634966845Z monitoring/prometheus/status/flags.stderr [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63515329Z namespaces/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63516153Z namespaces/cert-manager-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63516617Z namespaces/cert-manager-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635170331Z namespaces/cert-manager-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635203082Z namespaces/cert-manager-operator/coordination.k8s.io/leases/cert-manager-operator-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635292464Z namespaces/cert-manager-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635304614Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635355066Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635571382Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635811839Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.635978364Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636213751Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636438747Z namespaces/cert-manager-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636487048Z namespaces/cert-manager-operator/operators.coreos.com/installplans/install-mst96.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636628903Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636650333Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636723645Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636775907Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/openshift-cert-manager-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636838598Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63689231Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/openshift-cert-manager-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636957482Z namespaces/cert-manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636967892Z namespaces/cert-manager/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.636975162Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.637040264Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.637180148Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.637451626Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63758486Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.637812497Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638049463Z namespaces/default/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638074404Z namespaces/default/default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638172417Z namespaces/default/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638209858Z namespaces/default/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63829519Z namespaces/default/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638317791Z namespaces/default/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638417554Z namespaces/default/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638493366Z namespaces/default/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638579419Z namespaces/default/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638607149Z namespaces/default/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638655921Z namespaces/default/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638728173Z namespaces/default/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638774874Z namespaces/default/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638879487Z namespaces/default/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638938339Z namespaces/default/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.638964419Z namespaces/default/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639077053Z namespaces/default/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639145125Z namespaces/default/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639195166Z namespaces/default/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.63932838Z namespaces/default/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639428773Z namespaces/default/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639616268Z namespaces/default/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639700731Z namespaces/default/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639804654Z namespaces/default/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.639949558Z namespaces/default/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640112563Z namespaces/default/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640175744Z namespaces/default/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640221906Z namespaces/default/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640293478Z namespaces/default/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640342519Z namespaces/default/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640412011Z namespaces/default/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640461933Z namespaces/default/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640535724Z namespaces/default/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640616297Z namespaces/default/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640650058Z namespaces/default/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640696419Z namespaces/default/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64073928Z namespaces/default/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640780622Z namespaces/default/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640861284Z namespaces/default/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640878984Z namespaces/default/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.640888065Z namespaces/default/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641033939Z namespaces/default/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641213824Z namespaces/default/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641354338Z namespaces/default/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641556324Z namespaces/default/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641728249Z namespaces/default/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64176309Z namespaces/default/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641833102Z namespaces/default/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641877453Z namespaces/default/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641925515Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641932535Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641946225Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.641990716Z namespaces/kserve-ci-e2e-test/coordination.k8s.io/leases/epp-kserve-ci-e2e-test-scheduler-ha-replicas-test-inference-pool.llm-d.ai.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642044618Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642060188Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64210903Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642208932Z namespaces/kserve-ci-e2e-test/gateway.networking.k8s.io/gateways/router-gateway-2.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642277065Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642290225Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642331346Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/podmonitors/kserve-llm-isvc-vllm-engine-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642435509Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/podmonitors/kserve-llm-isvc-vllm-engine.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64248582Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642531792Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors/kserve-llm-isvc-scheduler-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642618344Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors/kserve-llm-isvc-scheduler.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642677866Z namespaces/kserve-ci-e2e-test/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642687386Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642744788Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.642913973Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.643092408Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.643223192Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.643455018Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.643688285Z namespaces/kserve/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.643735186Z namespaces/kserve/kserve.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64385098Z namespaces/kserve/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.643900581Z namespaces/kserve/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.643980653Z namespaces/kserve/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644022155Z namespaces/kserve/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644101967Z namespaces/kserve/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644320953Z namespaces/kserve/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644500278Z namespaces/kserve/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644541659Z namespaces/kserve/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644589401Z namespaces/kserve/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644638322Z namespaces/kserve/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644666883Z namespaces/kserve/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644753545Z namespaces/kserve/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644852768Z namespaces/kserve/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64490468Z namespaces/kserve/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.644989452Z namespaces/kserve/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645027333Z namespaces/kserve/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645034833Z namespaces/kserve/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645082825Z namespaces/kserve/coordination.k8s.io/leases/kserve-controller-manager-leader-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645168628Z namespaces/kserve/coordination.k8s.io/leases/llminferenceservice-kserve-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64525509Z namespaces/kserve/coordination.k8s.io/leases/odh-model-controller.opendatahub.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645371983Z namespaces/kserve/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645419225Z namespaces/kserve/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645652181Z namespaces/kserve/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.645757314Z namespaces/kserve/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.646083394Z namespaces/kserve/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.646162496Z namespaces/kserve/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.646423264Z namespaces/kserve/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.646548757Z namespaces/kserve/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64698902Z namespaces/kserve/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647078962Z namespaces/kserve/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647112813Z namespaces/kserve/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647227507Z namespaces/kserve/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647269828Z namespaces/kserve/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64734985Z namespaces/kserve/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647393671Z namespaces/kserve/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647481004Z namespaces/kserve/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647562446Z namespaces/kserve/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647619978Z namespaces/kserve/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647664179Z namespaces/kserve/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647724551Z namespaces/kserve/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647772682Z namespaces/kserve/monitoring.coreos.com/servicemonitors/model-serving-api-metrics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647884825Z namespaces/kserve/monitoring.coreos.com/servicemonitors/odh-model-controller-metrics-monitor.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647929416Z namespaces/kserve/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.647971628Z namespaces/kserve/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64802983Z namespaces/kserve/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64804643Z namespaces/kserve/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.648100492Z namespaces/kserve/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.648264836Z namespaces/kserve/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.648466802Z namespaces/kserve/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.648605956Z namespaces/kserve/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.648865643Z namespaces/kserve/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.6490767Z namespaces/kserve/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.64908436Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649135631Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/kserve-controller-manager-c8c545f5f-bn6xc.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649230424Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649238984Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649255245Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649278455Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649406939Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649474621Z namespaces/kserve/pods/kserve-controller-manager-c8c545f5f-bn6xc/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649525322Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649567074Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/llmisvc-controller-manager-859f687f98-95cs4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649661026Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649667916Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649674226Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.649723768Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651569861Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651639373Z namespaces/kserve/pods/llmisvc-controller-manager-859f687f98-95cs4/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651685234Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651731616Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/model-serving-api-fd65d7d6-sq9fg.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651850989Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/server/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651861449Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/server/server/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651869169Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/server/server/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.651922121Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/server/server/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652016414Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/server/server/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652084046Z namespaces/kserve/pods/model-serving-api-fd65d7d6-sq9fg/server/server/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652123517Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652163978Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/odh-model-controller-84b54b7f97-d2ssf.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652365464Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652375014Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652379174Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.652453236Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672155861Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672208283Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d2ssf/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672247284Z namespaces/kserve/pods/s3-init-lx5l4/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672294845Z namespaces/kserve/pods/s3-init-lx5l4/s3-init-lx5l4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672397228Z namespaces/kserve/pods/s3-init-lx5l4/download-hf-model/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672407078Z namespaces/kserve/pods/s3-init-lx5l4/download-hf-model/download-hf-model/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672421829Z namespaces/kserve/pods/s3-init-lx5l4/download-hf-model/download-hf-model/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672441519Z namespaces/kserve/pods/s3-init-lx5l4/download-hf-model/download-hf-model/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672530722Z namespaces/kserve/pods/s3-init-lx5l4/download-hf-model/download-hf-model/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672592674Z namespaces/kserve/pods/s3-init-lx5l4/download-hf-model/download-hf-model/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672611504Z namespaces/kserve/pods/s3-init-lx5l4/s3-init/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672624985Z namespaces/kserve/pods/s3-init-lx5l4/s3-init/s3-init/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672649265Z namespaces/kserve/pods/s3-init-lx5l4/s3-init/s3-init/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.672727138Z namespaces/kserve/pods/s3-init-lx5l4/s3-init/s3-init/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673047297Z namespaces/kserve/pods/s3-init-lx5l4/s3-init/s3-init/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673130739Z namespaces/kserve/pods/s3-init-lx5l4/s3-init/s3-init/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.6731758Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673223072Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/seaweedfs-64568bcd49-mwjrt.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673303564Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/seaweedfs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673313375Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/seaweedfs/seaweedfs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673317555Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/seaweedfs/seaweedfs/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673390297Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/seaweedfs/seaweedfs/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673561622Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/seaweedfs/seaweedfs/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673629133Z namespaces/kserve/pods/seaweedfs-64568bcd49-mwjrt/seaweedfs/seaweedfs/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673656694Z namespaces/kserve/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673704176Z namespaces/kserve/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673839319Z namespaces/kserve/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673889221Z namespaces/kserve/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.673981454Z namespaces/kuadrant-system/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674026875Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674105557Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674149299Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.67419167Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674234431Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674312893Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674608292Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.67489843Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674941211Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.674985452Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675060365Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675121846Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675209169Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.67524759Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675299472Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675381114Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675424715Z namespaces/kuadrant-system/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675433915Z namespaces/kuadrant-system/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675461556Z namespaces/kuadrant-system/coordination.k8s.io/leases/3745a16e.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675556159Z namespaces/kuadrant-system/coordination.k8s.io/leases/a3f98d6c.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675637321Z namespaces/kuadrant-system/coordination.k8s.io/leases/aac3a15d.authorino.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675721304Z namespaces/kuadrant-system/coordination.k8s.io/leases/f139389e.kuadrant.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675768945Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675810146Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.675974301Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.676090114Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.676498536Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.676602979Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.67697941Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677160685Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677380491Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677494624Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677537356Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677653989Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.67769462Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677760382Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677815344Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.677930867Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678011819Z namespaces/kuadrant-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678054Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678096092Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678142103Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678181944Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678255406Z namespaces/kuadrant-system/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678262446Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678313968Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678521464Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.67873878Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.678906185Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.679235644Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.679467801Z namespaces/kuadrant-system/operators.coreos.com/installplans/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.679515102Z namespaces/kuadrant-system/operators.coreos.com/installplans/install-kx8pn.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.679833391Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.679879543Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.679966135Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680050428Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.68013395Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680180341Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680225033Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/kuadrant.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680296955Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680345916Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/authorino-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680433209Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/dns-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680595363Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/limitador-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680688236Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/rhcl-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680777309Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680785699Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.68084634Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino-7c77957575-ddrbt.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680921693Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680931933Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680941613Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.680993775Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684100164Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684159855Z namespaces/kuadrant-system/pods/authorino-7c77957575-ddrbt/authorino/authorino/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684228428Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684265599Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/authorino-operator-6d85f6564-k7t49.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684367961Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684377042Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684386562Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684432603Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684611008Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.68467845Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-k7t49/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684717482Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684756273Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/dns-operator-controller-manager-65b49595d7-cklsz.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684880636Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684893507Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684912907Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.684938288Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685052241Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685121813Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-cklsz/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685153544Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685196455Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin-85fd4b8b9d-cpc85.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685270307Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685277848Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685282138Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685300428Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685384091Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685454653Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-cpc85/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685486324Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685533245Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685631618Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685638878Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685642888Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.685663739Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692275238Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.6923417Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-72b5j/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692391262Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692442193Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador-limitador-69574b596d-m97zt.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692539076Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692548036Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692553396Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692583107Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69268071Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692747762Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-m97zt/limitador/limitador/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692813204Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692866895Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/limitador-operator-controller-manager-6f9f468797-4hm5n.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692959168Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692976818Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.692983559Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693019949Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693162184Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693230466Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-4hm5n/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693271687Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693307728Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69336964Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693409261Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693464252Z namespaces/kube-node-lease/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693471102Z namespaces/kube-node-lease/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693477453Z namespaces/kube-node-lease/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693539444Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-129-204.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693635717Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-140-244.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69372579Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-142-38.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693770691Z namespaces/kube-node-lease/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693780251Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.693873224Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.694032349Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.694338498Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.694480422Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.694708868Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.694952925Z namespaces/kube-public/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.694965765Z namespaces/kube-public/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.694976646Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.695029567Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.695183252Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.695359337Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69549246Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.695694016Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.695962074Z namespaces/kube-system/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696008235Z namespaces/kube-system/kube-system.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696066657Z namespaces/kube-system/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696115649Z namespaces/kube-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69616711Z namespaces/kube-system/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69618752Z namespaces/kube-system/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696334885Z namespaces/kube-system/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696399426Z namespaces/kube-system/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696488489Z namespaces/kube-system/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69651121Z namespaces/kube-system/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696570062Z namespaces/kube-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696626343Z namespaces/kube-system/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696643124Z namespaces/kube-system/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696736726Z namespaces/kube-system/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696759287Z namespaces/kube-system/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696825509Z namespaces/kube-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696916831Z namespaces/kube-system/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696949652Z namespaces/kube-system/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.696961473Z namespaces/kube-system/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.697022434Z namespaces/kube-system/coordination.k8s.io/leases/apiserver-q5g3rtgrphbrukatp6vgc6x5ce.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.697136818Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-cainjector-leader-election.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.697233201Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.697339673Z namespaces/kube-system/coordination.k8s.io/leases/kube-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.697492768Z namespaces/kube-system/coordination.k8s.io/leases/kube-scheduler.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69757078Z namespaces/kube-system/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.697619512Z namespaces/kube-system/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.697806707Z namespaces/kube-system/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69790833Z namespaces/kube-system/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.698170597Z namespaces/kube-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.69825052Z namespaces/kube-system/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.698518687Z namespaces/kube-system/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.698655831Z namespaces/kube-system/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699006211Z namespaces/kube-system/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699079703Z namespaces/kube-system/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699136535Z namespaces/kube-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699225078Z namespaces/kube-system/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699290889Z namespaces/kube-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699341991Z namespaces/kube-system/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699380082Z namespaces/kube-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699466285Z namespaces/kube-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699545177Z namespaces/kube-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699591948Z namespaces/kube-system/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699628889Z namespaces/kube-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699682191Z namespaces/kube-system/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699718132Z namespaces/kube-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699766823Z namespaces/kube-system/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699778263Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.699855756Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70000298Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700179775Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700319169Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700506484Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70071552Z namespaces/kube-system/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700729811Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700766932Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/global-pull-secret-syncer-fkgrh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700927556Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/global-pull-secret-syncer/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700938847Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700943137Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.700960697Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701265206Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701353188Z namespaces/kube-system/pods/global-pull-secret-syncer-fkgrh/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70140626Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701451001Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/global-pull-secret-syncer-hrsrd.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701566355Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/global-pull-secret-syncer/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701573465Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701577315Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701607606Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.701933485Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702004317Z namespaces/kube-system/pods/global-pull-secret-syncer-hrsrd/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702035268Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702060479Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/global-pull-secret-syncer-xm895.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702141851Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/global-pull-secret-syncer/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702150502Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702160112Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702210423Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702515992Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702581554Z namespaces/kube-system/pods/global-pull-secret-syncer-xm895/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702617655Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702663196Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/konnectivity-agent-4sxrj.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702751309Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/konnectivity-agent/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702765529Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70278586Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.702836791Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.703579382Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.703647084Z namespaces/kube-system/pods/konnectivity-agent-4sxrj/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.703714886Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.703757988Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/konnectivity-agent-8nvsc.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70386304Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/konnectivity-agent/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.703876461Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.703880761Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.703911932Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.704843249Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.704917481Z namespaces/kube-system/pods/konnectivity-agent-8nvsc/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.704948132Z namespaces/kube-system/pods/konnectivity-agent-s5xls/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.704991653Z namespaces/kube-system/pods/konnectivity-agent-s5xls/konnectivity-agent-s5xls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.705080575Z namespaces/kube-system/pods/konnectivity-agent-s5xls/konnectivity-agent/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.705089226Z namespaces/kube-system/pods/konnectivity-agent-s5xls/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.705094346Z namespaces/kube-system/pods/konnectivity-agent-s5xls/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.705142487Z namespaces/kube-system/pods/konnectivity-agent-s5xls/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70593208Z namespaces/kube-system/pods/konnectivity-agent-s5xls/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706011242Z namespaces/kube-system/pods/konnectivity-agent-s5xls/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706056923Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706109095Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706189707Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/haproxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706196998Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706203238Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706245719Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706483836Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706564798Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-129-204.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70662786Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706663721Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706771974Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/haproxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706814175Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706825125Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.706838726Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707100553Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707166075Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-140-244.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707198606Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707240357Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7073188Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/haproxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70732756Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70733164Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707371971Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707584907Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707652779Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-142-38.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70769274Z namespaces/kube-system/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707722121Z namespaces/kube-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707766583Z namespaces/kube-system/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707775683Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707854075Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/csi-snapshot-controller-operator-authentication-reader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.707948098Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70800905Z namespaces/kube-system/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708054191Z namespaces/kube-system/rbac.authorization.k8s.io/roles/extension-apiserver-authentication-reader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708104742Z namespaces/kube-system/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708146333Z namespaces/kube-system/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708193045Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708200245Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708206805Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708270367Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708460872Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708659398Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.708805232Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.70905754Z namespaces/open-cluster-management-6eeb9ecb-e130-4fe7-acae-0164835f2/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709266675Z namespaces/open-cluster-management-agent-addon/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709274176Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709278076Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709306437Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/cluster-proxy.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709401489Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/work-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709445301Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709465041Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709514983Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709667807Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.709870243Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710009457Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710223773Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710443919Z namespaces/openshift-apiserver-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710495671Z namespaces/openshift-apiserver-operator/openshift-apiserver-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710569033Z namespaces/openshift-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710612394Z namespaces/openshift-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710681696Z namespaces/openshift-apiserver-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710722777Z namespaces/openshift-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7108092Z namespaces/openshift-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710913893Z namespaces/openshift-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.710981605Z namespaces/openshift-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711036566Z namespaces/openshift-apiserver-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711061957Z namespaces/openshift-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711116719Z namespaces/openshift-apiserver-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711137209Z namespaces/openshift-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711229832Z namespaces/openshift-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711278843Z namespaces/openshift-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711325325Z namespaces/openshift-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711403667Z namespaces/openshift-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711446658Z namespaces/openshift-apiserver-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.711489239Z namespaces/openshift-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.712456677Z namespaces/openshift-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.712539519Z namespaces/openshift-apiserver-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.712629532Z namespaces/openshift-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.712706584Z namespaces/openshift-apiserver-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.712808987Z namespaces/openshift-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.712957021Z namespaces/openshift-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713159937Z namespaces/openshift-apiserver-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713294501Z namespaces/openshift-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713343772Z namespaces/openshift-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713381013Z namespaces/openshift-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713424225Z namespaces/openshift-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713467906Z namespaces/openshift-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713510907Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71360193Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713681192Z namespaces/openshift-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713724533Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713765664Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713861547Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713897688Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/openshift-apiserver-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71395426Z namespaces/openshift-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.713995121Z namespaces/openshift-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714039042Z namespaces/openshift-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714049933Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714096994Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714247058Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714418343Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714561687Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714784764Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.714989139Z namespaces/openshift-apiserver-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715029841Z namespaces/openshift-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715087752Z namespaces/openshift-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715115693Z namespaces/openshift-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715211166Z namespaces/openshift-apiserver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715254207Z namespaces/openshift-apiserver/openshift-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71533538Z namespaces/openshift-apiserver/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715379751Z namespaces/openshift-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715425902Z namespaces/openshift-apiserver/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715465393Z namespaces/openshift-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715544506Z namespaces/openshift-apiserver/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715617908Z namespaces/openshift-apiserver/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7156993Z namespaces/openshift-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715744851Z namespaces/openshift-apiserver/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715784893Z namespaces/openshift-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715862605Z namespaces/openshift-apiserver/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715903506Z namespaces/openshift-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.715991358Z namespaces/openshift-apiserver/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71604874Z namespaces/openshift-apiserver/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716094911Z namespaces/openshift-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716185224Z namespaces/openshift-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716265936Z namespaces/openshift-apiserver/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716311868Z namespaces/openshift-apiserver/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716421231Z namespaces/openshift-apiserver/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716524464Z namespaces/openshift-apiserver/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716610636Z namespaces/openshift-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716708459Z namespaces/openshift-apiserver/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.716802702Z namespaces/openshift-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717017208Z namespaces/openshift-apiserver/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717181633Z namespaces/openshift-apiserver/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717234354Z namespaces/openshift-apiserver/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717281095Z namespaces/openshift-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717341927Z namespaces/openshift-apiserver/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717371628Z namespaces/openshift-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71743908Z namespaces/openshift-apiserver/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717487501Z namespaces/openshift-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717586394Z namespaces/openshift-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717672916Z namespaces/openshift-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717735228Z namespaces/openshift-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717770369Z namespaces/openshift-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717857942Z namespaces/openshift-apiserver/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717919314Z namespaces/openshift-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717981916Z namespaces/openshift-apiserver/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.717991636Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.718055858Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.718207082Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.718383027Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.718667345Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.718898602Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719087697Z namespaces/openshift-apiserver/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719130128Z namespaces/openshift-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719173339Z namespaces/openshift-apiserver/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719230361Z namespaces/openshift-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71952453Z namespaces/openshift-authentication-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7195348Z namespaces/openshift-authentication-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71953956Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719591192Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/authentication-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719646013Z namespaces/openshift-authentication-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719657453Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.719714915Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.71988848Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720066425Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720198259Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720424246Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720611851Z namespaces/openshift-authentication/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720621051Z namespaces/openshift-authentication/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720629111Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720686903Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.720850688Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721029413Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721161697Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721347892Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721530937Z namespaces/openshift-cloud-controller-manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721539718Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721544648Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721586949Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/cloud-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721657931Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721666281Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721716063Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.721887957Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722053962Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722191326Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722389392Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722605258Z namespaces/openshift-cloud-credential-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722627229Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722631979Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722642849Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/cloud-credential-operator-leader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722704301Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722717451Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722747392Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/cloud-credential-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722834775Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722851425Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.722896626Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.723049491Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.723221766Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72336789Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.723596737Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.723807703Z namespaces/openshift-cloud-network-config-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.723865664Z namespaces/openshift-cloud-network-config-controller/openshift-cloud-network-config-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.723921756Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.723966017Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724039609Z namespaces/openshift-cloud-network-config-controller/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72407652Z namespaces/openshift-cloud-network-config-controller/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724162583Z namespaces/openshift-cloud-network-config-controller/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724237535Z namespaces/openshift-cloud-network-config-controller/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724316737Z namespaces/openshift-cloud-network-config-controller/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724374039Z namespaces/openshift-cloud-network-config-controller/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724388089Z namespaces/openshift-cloud-network-config-controller/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724444681Z namespaces/openshift-cloud-network-config-controller/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724487372Z namespaces/openshift-cloud-network-config-controller/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724566484Z namespaces/openshift-cloud-network-config-controller/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724632136Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724660417Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724745749Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72476843Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724802871Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.724922904Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/cloud-network-config-controller-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725015277Z namespaces/openshift-cloud-network-config-controller/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725170682Z namespaces/openshift-cloud-network-config-controller/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725183252Z namespaces/openshift-cloud-network-config-controller/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725240273Z namespaces/openshift-cloud-network-config-controller/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725319326Z namespaces/openshift-cloud-network-config-controller/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725391298Z namespaces/openshift-cloud-network-config-controller/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725469Z namespaces/openshift-cloud-network-config-controller/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725613194Z namespaces/openshift-cloud-network-config-controller/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725718427Z namespaces/openshift-cloud-network-config-controller/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725732788Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72582588Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/cloud-network-config-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725857351Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725907743Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725930813Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.725984735Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726073848Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726092798Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72618118Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726258393Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726306134Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726348525Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726395807Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726428938Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726479259Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72649524Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72650901Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726683055Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.726887101Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727028045Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72722501Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727412296Z namespaces/openshift-cloud-network-config-controller/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727452757Z namespaces/openshift-cloud-network-config-controller/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727493428Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727500928Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72755332Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/cloud-network-config-controller-rb.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727586081Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727638522Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/cloud-network-config-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727678774Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727710034Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727807747Z namespaces/openshift-cluster-csi-drivers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727861319Z namespaces/openshift-cluster-csi-drivers/openshift-cluster-csi-drivers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727929121Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.727977662Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728041204Z namespaces/openshift-cluster-csi-drivers/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728083555Z namespaces/openshift-cluster-csi-drivers/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728210209Z namespaces/openshift-cluster-csi-drivers/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728287261Z namespaces/openshift-cluster-csi-drivers/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728369273Z namespaces/openshift-cluster-csi-drivers/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728415465Z namespaces/openshift-cluster-csi-drivers/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728463676Z namespaces/openshift-cluster-csi-drivers/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728522958Z namespaces/openshift-cluster-csi-drivers/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728561119Z namespaces/openshift-cluster-csi-drivers/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728642691Z namespaces/openshift-cluster-csi-drivers/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728702123Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728744004Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728850107Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728902569Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.728913689Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72895219Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ebs-csi-aws-com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729040142Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-attacher-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729128085Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-resizer-ebs-csi-aws-com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729251849Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-snapshotter-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729346211Z namespaces/openshift-cluster-csi-drivers/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729387083Z namespaces/openshift-cluster-csi-drivers/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729487655Z namespaces/openshift-cluster-csi-drivers/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729564568Z namespaces/openshift-cluster-csi-drivers/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.729910078Z namespaces/openshift-cluster-csi-drivers/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.72998945Z namespaces/openshift-cluster-csi-drivers/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730195376Z namespaces/openshift-cluster-csi-drivers/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73033125Z namespaces/openshift-cluster-csi-drivers/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730472294Z namespaces/openshift-cluster-csi-drivers/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730512275Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730556046Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/aws-ebs-csi-driver-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730610678Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730640439Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73069565Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730724221Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730782023Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730846974Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.730928587Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73102521Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731107662Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731136693Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731204205Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731261786Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731354239Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731364959Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731419511Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731569515Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7317487Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.731909435Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73210372Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732454731Z namespaces/openshift-cluster-csi-drivers/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732462801Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732516812Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/aws-ebs-csi-driver-node-bkpvw.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732632085Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-driver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732648126Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-driver/csi-driver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732652616Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-driver/csi-driver/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732686787Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73278306Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732904243Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732922284Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-liveness-probe/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732927334Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.732942205Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733002716Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733093169Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733171031Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733336146Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-node-driver-registrar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733344086Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733353666Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733408088Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733507221Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733577973Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-bkpvw/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733625954Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733672585Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/aws-ebs-csi-driver-node-hcfc4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733761778Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-driver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733772768Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-driver/csi-driver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733781909Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-driver/csi-driver/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.733946583Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734044696Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734115138Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734157539Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-liveness-probe/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734164529Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73417106Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734228111Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734303444Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734377746Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734417187Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-node-driver-registrar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734429547Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734449038Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734471258Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734568581Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734632053Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-hcfc4/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734691535Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734734696Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/aws-ebs-csi-driver-node-jqhpm.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734844189Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-driver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734858069Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-driver/csi-driver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73487487Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-driver/csi-driver/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.734928902Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735029774Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735086096Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735142757Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-liveness-probe/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735150418Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735154628Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735206659Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735300382Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735369954Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735418636Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-node-driver-registrar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735427106Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735434386Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735473767Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73557388Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735644212Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-jqhpm/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735690173Z namespaces/openshift-cluster-csi-drivers/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735719104Z namespaces/openshift-cluster-csi-drivers/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735804687Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735816087Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735860448Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/aws-ebs-csi-driver-operator-rolebinding.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73593149Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.735960071Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/aws-ebs-csi-driver-operator-role.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736020653Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736068754Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736125696Z namespaces/openshift-cluster-machine-approver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736133256Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736137336Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736182717Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/cluster-machine-approver-leader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736245409Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73625974Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73628527Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/cluster-machine-approver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736348692Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736359003Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736409214Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736560198Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736737243Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.736890108Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737103054Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737301899Z namespaces/openshift-cluster-node-tuning-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737349481Z namespaces/openshift-cluster-node-tuning-operator/openshift-cluster-node-tuning-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737462654Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737506345Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737581298Z namespaces/openshift-cluster-node-tuning-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737628039Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737749642Z namespaces/openshift-cluster-node-tuning-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737854375Z namespaces/openshift-cluster-node-tuning-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737928797Z namespaces/openshift-cluster-node-tuning-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.737988599Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73800365Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/tuned.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738107633Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738125363Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738194375Z namespaces/openshift-cluster-node-tuning-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738236186Z namespaces/openshift-cluster-node-tuning-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738316779Z namespaces/openshift-cluster-node-tuning-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73836293Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738391141Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738482333Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738529415Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738542385Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738585056Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/node-tuning-operator-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738666378Z namespaces/openshift-cluster-node-tuning-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.738696019Z namespaces/openshift-cluster-node-tuning-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.73978221Z namespaces/openshift-cluster-node-tuning-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.739927665Z namespaces/openshift-cluster-node-tuning-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.740084209Z namespaces/openshift-cluster-node-tuning-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.740192272Z namespaces/openshift-cluster-node-tuning-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.740430669Z namespaces/openshift-cluster-node-tuning-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.740615025Z namespaces/openshift-cluster-node-tuning-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74080874Z namespaces/openshift-cluster-node-tuning-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741009246Z namespaces/openshift-cluster-node-tuning-operator/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741075128Z namespaces/openshift-cluster-node-tuning-operator/core/serviceaccounts/cluster-node-tuning-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74115708Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741211782Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741290214Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741347635Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741430888Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741488859Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741602083Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741711096Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741811009Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.741999044Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742081067Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742142838Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/node-tuning-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742239701Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742296573Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/node-tuning-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742370565Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742422746Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742492318Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742523209Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742596491Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.742857249Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.743102146Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.743268961Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.743575609Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.743847047Z namespaces/openshift-cluster-node-tuning-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74395924Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744030652Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/tuned-bkhjm.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744160886Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/tuned/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744175136Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/tuned/tuned/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744185147Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/tuned/tuned/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744259879Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/tuned/tuned/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744413403Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744497296Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-bkhjm/tuned/tuned/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744576748Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74462678Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/tuned-vgpjk.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744710222Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/tuned/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744719182Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/tuned/tuned/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744724032Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/tuned/tuned/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744754603Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/tuned/tuned/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744886257Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.744956969Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-vgpjk/tuned/tuned/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7449806Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745032221Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/tuned-zln49.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745113183Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/tuned/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745120934Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/tuned/tuned/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745124684Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/tuned/tuned/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745158825Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/tuned/tuned/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745252547Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74532931Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-zln49/tuned/tuned/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745364731Z namespaces/openshift-cluster-node-tuning-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745401522Z namespaces/openshift-cluster-node-tuning-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745444573Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745490944Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745538096Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745783713Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745846944Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-129-204.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.745949667Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-140-244.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74603168Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-142-38.ec2.internal.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746080761Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746109842Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746212195Z namespaces/openshift-cluster-samples-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746261786Z namespaces/openshift-cluster-samples-operator/openshift-cluster-samples-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746456952Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746491563Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746559365Z namespaces/openshift-cluster-samples-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746595186Z namespaces/openshift-cluster-samples-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746683648Z namespaces/openshift-cluster-samples-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746811742Z namespaces/openshift-cluster-samples-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746932216Z namespaces/openshift-cluster-samples-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.746977507Z namespaces/openshift-cluster-samples-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747043189Z namespaces/openshift-cluster-samples-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74708067Z namespaces/openshift-cluster-samples-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747118751Z namespaces/openshift-cluster-samples-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747205274Z namespaces/openshift-cluster-samples-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747266335Z namespaces/openshift-cluster-samples-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747318317Z namespaces/openshift-cluster-samples-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747401719Z namespaces/openshift-cluster-samples-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747477411Z namespaces/openshift-cluster-samples-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747517152Z namespaces/openshift-cluster-samples-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747732319Z namespaces/openshift-cluster-samples-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747842902Z namespaces/openshift-cluster-samples-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.747967185Z namespaces/openshift-cluster-samples-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748043838Z namespaces/openshift-cluster-samples-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748164311Z namespaces/openshift-cluster-samples-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748286785Z namespaces/openshift-cluster-samples-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74847075Z namespaces/openshift-cluster-samples-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748520741Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748559222Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748632564Z namespaces/openshift-cluster-samples-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748672516Z namespaces/openshift-cluster-samples-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748745408Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748783959Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748891492Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748979764Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.748998635Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749058007Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749113258Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.74916401Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/samples-operator-alerts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749246882Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749286993Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/cluster-samples-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749335575Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749385296Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749429537Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749436647Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749460898Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749619133Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.749812188Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750095336Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750310032Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750494838Z namespaces/openshift-cluster-samples-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750502078Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750539379Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator-56fcf86bf-mrxwp.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750647572Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator-watch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750654932Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator-watch/cluster-samples-operator-watch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750662282Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750716284Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750842088Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75092063Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750964831Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.750975551Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator/cluster-samples-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.751003692Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator/cluster-samples-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.751010863Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator/cluster-samples-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.751823606Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator/cluster-samples-operator/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.751894878Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-56fcf86bf-mrxwp/cluster-samples-operator/cluster-samples-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.751938079Z namespaces/openshift-cluster-samples-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75198908Z namespaces/openshift-cluster-samples-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752050482Z namespaces/openshift-cluster-samples-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752092143Z namespaces/openshift-cluster-samples-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752182946Z namespaces/openshift-cluster-storage-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752236588Z namespaces/openshift-cluster-storage-operator/openshift-cluster-storage-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752300189Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752343401Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752417163Z namespaces/openshift-cluster-storage-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752500585Z namespaces/openshift-cluster-storage-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752620909Z namespaces/openshift-cluster-storage-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752741732Z namespaces/openshift-cluster-storage-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752862706Z namespaces/openshift-cluster-storage-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752899637Z namespaces/openshift-cluster-storage-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.752949458Z namespaces/openshift-cluster-storage-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75301446Z namespaces/openshift-cluster-storage-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753079952Z namespaces/openshift-cluster-storage-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753179915Z namespaces/openshift-cluster-storage-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753243327Z namespaces/openshift-cluster-storage-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753268537Z namespaces/openshift-cluster-storage-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75336946Z namespaces/openshift-cluster-storage-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753411431Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753423202Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753470493Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/data-source-validator-leader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753556736Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/snapshot-controller-leader.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753637448Z namespaces/openshift-cluster-storage-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753687489Z namespaces/openshift-cluster-storage-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753783652Z namespaces/openshift-cluster-storage-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.753899925Z namespaces/openshift-cluster-storage-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754187724Z namespaces/openshift-cluster-storage-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754260226Z namespaces/openshift-cluster-storage-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754370409Z namespaces/openshift-cluster-storage-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754533964Z namespaces/openshift-cluster-storage-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754681508Z namespaces/openshift-cluster-storage-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75474647Z namespaces/openshift-cluster-storage-operator/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754805601Z namespaces/openshift-cluster-storage-operator/core/serviceaccounts/csi-snapshot-controller-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754884134Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754923395Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.754977376Z namespaces/openshift-cluster-storage-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755021327Z namespaces/openshift-cluster-storage-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75509337Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755136111Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755220583Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755310736Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755372628Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755421869Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75547164Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755509341Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/prometheus.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755611404Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755649956Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/cluster-storage-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755721868Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755770919Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755880452Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755898393Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.755922973Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756069818Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756245803Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756385597Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756575562Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756750667Z namespaces/openshift-cluster-storage-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756758837Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756814499Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/volume-data-source-validator-5d456fdbcd-ldf6s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756899232Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/volume-data-source-validator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756906562Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/volume-data-source-validator/volume-data-source-validator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756910862Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/volume-data-source-validator/volume-data-source-validator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.756947873Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/volume-data-source-validator/volume-data-source-validator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757047516Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/volume-data-source-validator/volume-data-source-validator/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757116258Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-5d456fdbcd-ldf6s/volume-data-source-validator/volume-data-source-validator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757133008Z namespaces/openshift-cluster-storage-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75718572Z namespaces/openshift-cluster-storage-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757249911Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757257212Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757300373Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/rolebindings/csi-snapshot-controller-operator-role.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757355565Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757397296Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/roles/csi-snapshot-controller-operator-role.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757442767Z namespaces/openshift-cluster-storage-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757489788Z namespaces/openshift-cluster-storage-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757563111Z namespaces/openshift-cluster-version/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757611272Z namespaces/openshift-cluster-version/openshift-cluster-version.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757679494Z namespaces/openshift-cluster-version/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757719145Z namespaces/openshift-cluster-version/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757785627Z namespaces/openshift-cluster-version/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.757946442Z namespaces/openshift-cluster-version/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758122187Z namespaces/openshift-cluster-version/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758199479Z namespaces/openshift-cluster-version/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758282071Z namespaces/openshift-cluster-version/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758315982Z namespaces/openshift-cluster-version/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758363043Z namespaces/openshift-cluster-version/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758434696Z namespaces/openshift-cluster-version/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758484057Z namespaces/openshift-cluster-version/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75857698Z namespaces/openshift-cluster-version/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758642451Z namespaces/openshift-cluster-version/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758681663Z namespaces/openshift-cluster-version/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758778575Z namespaces/openshift-cluster-version/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758853048Z namespaces/openshift-cluster-version/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758861468Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.758899439Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/version.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759013222Z namespaces/openshift-cluster-version/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759061374Z namespaces/openshift-cluster-version/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759167397Z namespaces/openshift-cluster-version/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759253899Z namespaces/openshift-cluster-version/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759388073Z namespaces/openshift-cluster-version/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759476255Z namespaces/openshift-cluster-version/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759549797Z namespaces/openshift-cluster-version/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759674351Z namespaces/openshift-cluster-version/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759894227Z namespaces/openshift-cluster-version/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.759973569Z namespaces/openshift-cluster-version/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.75998969Z namespaces/openshift-cluster-version/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760059042Z namespaces/openshift-cluster-version/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760107664Z namespaces/openshift-cluster-version/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760168095Z namespaces/openshift-cluster-version/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760202756Z namespaces/openshift-cluster-version/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760276908Z namespaces/openshift-cluster-version/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760362131Z namespaces/openshift-cluster-version/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760411492Z namespaces/openshift-cluster-version/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760432263Z namespaces/openshift-cluster-version/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760489824Z namespaces/openshift-cluster-version/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760526336Z namespaces/openshift-cluster-version/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760592127Z namespaces/openshift-cluster-version/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760606328Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760654999Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.760810604Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761006659Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761146053Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761346379Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761524974Z namespaces/openshift-cluster-version/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761542985Z namespaces/openshift-cluster-version/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761621367Z namespaces/openshift-cluster-version/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761654728Z namespaces/openshift-cluster-version/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761762811Z namespaces/openshift-config-managed/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.761817173Z namespaces/openshift-config-managed/openshift-config-managed.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762015868Z namespaces/openshift-config-managed/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762040389Z namespaces/openshift-config-managed/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762128782Z namespaces/openshift-config-managed/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762150342Z namespaces/openshift-config-managed/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762249525Z namespaces/openshift-config-managed/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762322597Z namespaces/openshift-config-managed/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762402069Z namespaces/openshift-config-managed/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.76244642Z namespaces/openshift-config-managed/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762467601Z namespaces/openshift-config-managed/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762550223Z namespaces/openshift-config-managed/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762587815Z namespaces/openshift-config-managed/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762670527Z namespaces/openshift-config-managed/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762720619Z namespaces/openshift-config-managed/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.76277087Z namespaces/openshift-config-managed/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762885673Z namespaces/openshift-config-managed/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.762966035Z namespaces/openshift-config-managed/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.763002167Z namespaces/openshift-config-managed/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.76660757Z namespaces/openshift-config-managed/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.766666381Z namespaces/openshift-config-managed/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.766752754Z namespaces/openshift-config-managed/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.766874947Z namespaces/openshift-config-managed/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.766985811Z namespaces/openshift-config-managed/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767136025Z namespaces/openshift-config-managed/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767231908Z namespaces/openshift-config-managed/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767293339Z namespaces/openshift-config-managed/core/configmaps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767351911Z namespaces/openshift-config-managed/core/configmaps/console-public.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767427283Z namespaces/openshift-config-managed/core/configmaps/openshift-network-features.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767497665Z namespaces/openshift-config-managed/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767530846Z namespaces/openshift-config-managed/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767589728Z namespaces/openshift-config-managed/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.76765172Z namespaces/openshift-config-managed/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767713931Z namespaces/openshift-config-managed/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767749353Z namespaces/openshift-config-managed/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767863116Z namespaces/openshift-config-managed/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.767944898Z namespaces/openshift-config-managed/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.76801541Z namespaces/openshift-config-managed/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768055461Z namespaces/openshift-config-managed/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768121883Z namespaces/openshift-config-managed/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768158324Z namespaces/openshift-config-managed/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768207596Z namespaces/openshift-config-managed/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768216896Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768269767Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768590707Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768819143Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.768973588Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769176544Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.76938964Z namespaces/openshift-config-managed/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769431011Z namespaces/openshift-config-managed/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769501503Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769519683Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769549384Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/openshift-network-public-role-binding.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769607246Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769649547Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/openshift-network-public-role.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769691068Z namespaces/openshift-config-managed/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769727769Z namespaces/openshift-config-managed/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769859203Z namespaces/openshift-config-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769871953Z namespaces/openshift-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769877914Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.769983677Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/config-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.770034798Z namespaces/openshift-config-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.770055269Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77011195Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.770256504Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77044073Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.770577344Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77080675Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771017026Z namespaces/openshift-config/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771058978Z namespaces/openshift-config/openshift-config.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771253573Z namespaces/openshift-config/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771319775Z namespaces/openshift-config/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771392517Z namespaces/openshift-config/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771419878Z namespaces/openshift-config/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77150142Z namespaces/openshift-config/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771582613Z namespaces/openshift-config/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771659555Z namespaces/openshift-config/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771707406Z namespaces/openshift-config/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771739657Z namespaces/openshift-config/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771806299Z namespaces/openshift-config/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77185735Z namespaces/openshift-config/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771951763Z namespaces/openshift-config/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.771975864Z namespaces/openshift-config/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772029505Z namespaces/openshift-config/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772112768Z namespaces/openshift-config/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772136238Z namespaces/openshift-config/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77218802Z namespaces/openshift-config/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772309783Z namespaces/openshift-config/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772384925Z namespaces/openshift-config/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772469638Z namespaces/openshift-config/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77254118Z namespaces/openshift-config/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772617472Z namespaces/openshift-config/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772756276Z namespaces/openshift-config/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772908731Z namespaces/openshift-config/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.772965362Z namespaces/openshift-config/core/secrets/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773008083Z namespaces/openshift-config/core/secrets/pull-secret.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773070445Z namespaces/openshift-config/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773108366Z namespaces/openshift-config/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773150327Z namespaces/openshift-config/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773189729Z namespaces/openshift-config/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77324707Z namespaces/openshift-config/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773286821Z namespaces/openshift-config/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773365974Z namespaces/openshift-config/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773445036Z namespaces/openshift-config/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773502488Z namespaces/openshift-config/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773546609Z namespaces/openshift-config/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77359656Z namespaces/openshift-config/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773637692Z namespaces/openshift-config/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773697503Z namespaces/openshift-config/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773707214Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.773738034Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77391628Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.774166407Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7743021Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.774490316Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.774678981Z namespaces/openshift-config/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.774697282Z namespaces/openshift-config/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.774765604Z namespaces/openshift-config/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.774814825Z namespaces/openshift-config/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775004051Z namespaces/openshift-console-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775053812Z namespaces/openshift-console-operator/openshift-console-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775205666Z namespaces/openshift-console-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775230547Z namespaces/openshift-console-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775310849Z namespaces/openshift-console-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77533197Z namespaces/openshift-console-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775440003Z namespaces/openshift-console-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775567957Z namespaces/openshift-console-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77567953Z namespaces/openshift-console-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775718891Z namespaces/openshift-console-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775770733Z namespaces/openshift-console-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775826084Z namespaces/openshift-console-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775872896Z namespaces/openshift-console-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775951028Z namespaces/openshift-console-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.775999149Z namespaces/openshift-console-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77602178Z namespaces/openshift-console-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.776110332Z namespaces/openshift-console-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.776158004Z namespaces/openshift-console-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.776168794Z namespaces/openshift-console-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.776212085Z namespaces/openshift-console-operator/coordination.k8s.io/leases/console-operator-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.776299548Z namespaces/openshift-console-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.776348049Z namespaces/openshift-console-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.776948846Z namespaces/openshift-console-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777034069Z namespaces/openshift-console-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777278716Z namespaces/openshift-console-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777350668Z namespaces/openshift-console-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777477462Z namespaces/openshift-console-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777598085Z namespaces/openshift-console-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77776641Z namespaces/openshift-console-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777840042Z namespaces/openshift-console-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777892543Z namespaces/openshift-console-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.777961925Z namespaces/openshift-console-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778002957Z namespaces/openshift-console-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778089139Z namespaces/openshift-console-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77811419Z namespaces/openshift-console-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778216713Z namespaces/openshift-console-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778286745Z namespaces/openshift-console-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778359687Z namespaces/openshift-console-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778382697Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77847536Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778522841Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/cluster-monitoring-prometheus-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778577553Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778630615Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/console-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778692376Z namespaces/openshift-console-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778713467Z namespaces/openshift-console-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778771879Z namespaces/openshift-console-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.77880577Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.778861711Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779008015Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779184311Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779334225Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779544641Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779720466Z namespaces/openshift-console-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779730216Z namespaces/openshift-console-operator/pods/console-operator-68b757fb67-2s4gv/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779812269Z namespaces/openshift-console-operator/pods/console-operator-68b757fb67-2s4gv/console-operator-68b757fb67-2s4gv.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779981173Z namespaces/openshift-console-operator/pods/console-operator-68b757fb67-2s4gv/console-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779990954Z namespaces/openshift-console-operator/pods/console-operator-68b757fb67-2s4gv/console-operator/console-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.779995214Z namespaces/openshift-console-operator/pods/console-operator-68b757fb67-2s4gv/console-operator/console-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780047645Z namespaces/openshift-console-operator/pods/console-operator-68b757fb67-2s4gv/console-operator/console-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780538699Z namespaces/openshift-console-operator/pods/console-operator-68b757fb67-2s4gv/console-operator/console-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780594691Z namespaces/openshift-console-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780625212Z namespaces/openshift-console-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780691214Z namespaces/openshift-console-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780771996Z namespaces/openshift-console-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780863689Z namespaces/openshift-console-user-settings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780877839Z namespaces/openshift-console-user-settings/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780889509Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.780935321Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.781102215Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7812754Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.781415224Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78160928Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.782012872Z namespaces/openshift-console/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.782057683Z namespaces/openshift-console/openshift-console.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.782268259Z namespaces/openshift-console/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78228778Z namespaces/openshift-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.782368712Z namespaces/openshift-console/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.782418703Z namespaces/openshift-console/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.782510486Z namespaces/openshift-console/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78266723Z namespaces/openshift-console/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.782965299Z namespaces/openshift-console/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783024801Z namespaces/openshift-console/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783081562Z namespaces/openshift-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783140864Z namespaces/openshift-console/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783180355Z namespaces/openshift-console/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783256787Z namespaces/openshift-console/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78334237Z namespaces/openshift-console/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783403092Z namespaces/openshift-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783497484Z namespaces/openshift-console/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783556516Z namespaces/openshift-console/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.783592067Z namespaces/openshift-console/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.784185474Z namespaces/openshift-console/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.784277526Z namespaces/openshift-console/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.784541434Z namespaces/openshift-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.784614826Z namespaces/openshift-console/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.784841583Z namespaces/openshift-console/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.784980667Z namespaces/openshift-console/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785167012Z namespaces/openshift-console/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785248805Z namespaces/openshift-console/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785297676Z namespaces/openshift-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785375458Z namespaces/openshift-console/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785425819Z namespaces/openshift-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785483081Z namespaces/openshift-console/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785533622Z namespaces/openshift-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785613475Z namespaces/openshift-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785691707Z namespaces/openshift-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785749809Z namespaces/openshift-console/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785769739Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785877943Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785899503Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/console.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.785980715Z namespaces/openshift-console/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786014106Z namespaces/openshift-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786077468Z namespaces/openshift-console/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786089518Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786117519Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786271894Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786455379Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786597543Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.786785769Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787011255Z namespaces/openshift-console/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787019315Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787057836Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/console-68967f8d8c-jm6jg.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787164139Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/console/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78717284Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/console/console/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78717699Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/console/console/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787222261Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/console/console/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787336714Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/console/console/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787404076Z namespaces/openshift-console/pods/console-68967f8d8c-jm6jg/console/console/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787432057Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787478158Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/downloads-685bfb5f64-hkhkw.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787569431Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/download-server/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787577311Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/download-server/download-server/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787581251Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/download-server/download-server/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.787613422Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/download-server/download-server/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788004264Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/download-server/download-server/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788053925Z namespaces/openshift-console/pods/downloads-685bfb5f64-hkhkw/download-server/download-server/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788098976Z namespaces/openshift-console/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788133427Z namespaces/openshift-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788210019Z namespaces/openshift-console/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78822875Z namespaces/openshift-console/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788353443Z namespaces/openshift-controller-manager-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788372694Z namespaces/openshift-controller-manager-operator/openshift-controller-manager-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788440916Z namespaces/openshift-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788496518Z namespaces/openshift-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788558989Z namespaces/openshift-controller-manager-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.78857755Z namespaces/openshift-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788666843Z namespaces/openshift-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788745655Z namespaces/openshift-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788854058Z namespaces/openshift-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788882079Z namespaces/openshift-controller-manager-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.788952441Z namespaces/openshift-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789004482Z namespaces/openshift-controller-manager-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789055834Z namespaces/openshift-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789125206Z namespaces/openshift-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789183197Z namespaces/openshift-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789227448Z namespaces/openshift-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789315501Z namespaces/openshift-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789540988Z namespaces/openshift-controller-manager-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789591679Z namespaces/openshift-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789716502Z namespaces/openshift-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.789815435Z namespaces/openshift-controller-manager-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790029251Z namespaces/openshift-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790105004Z namespaces/openshift-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790187936Z namespaces/openshift-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790305829Z namespaces/openshift-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790463984Z namespaces/openshift-controller-manager-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790522905Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790563387Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790628008Z namespaces/openshift-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79066353Z namespaces/openshift-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790723721Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790742972Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790876756Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790956138Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.790999589Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791041351Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791080861Z namespaces/openshift-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791130513Z namespaces/openshift-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791173934Z namespaces/openshift-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791185865Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791214905Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79137904Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791572416Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79172761Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.791976377Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792166603Z namespaces/openshift-controller-manager-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792212684Z namespaces/openshift-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792279366Z namespaces/openshift-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792310097Z namespaces/openshift-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79240618Z namespaces/openshift-controller-manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792447721Z namespaces/openshift-controller-manager/openshift-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792502742Z namespaces/openshift-controller-manager/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792543943Z namespaces/openshift-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792616846Z namespaces/openshift-controller-manager/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792631486Z namespaces/openshift-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792733459Z namespaces/openshift-controller-manager/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792808301Z namespaces/openshift-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792913224Z namespaces/openshift-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.792970066Z namespaces/openshift-controller-manager/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793014227Z namespaces/openshift-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793062649Z namespaces/openshift-controller-manager/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.7931039Z namespaces/openshift-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793182552Z namespaces/openshift-controller-manager/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793243064Z namespaces/openshift-controller-manager/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793293055Z namespaces/openshift-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793370207Z namespaces/openshift-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793572053Z namespaces/openshift-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793582443Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793627325Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793706137Z namespaces/openshift-controller-manager/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793748198Z namespaces/openshift-controller-manager/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793892692Z namespaces/openshift-controller-manager/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.793965274Z namespaces/openshift-controller-manager/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794057007Z namespaces/openshift-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794134659Z namespaces/openshift-controller-manager/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794217431Z namespaces/openshift-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794350845Z namespaces/openshift-controller-manager/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79451697Z namespaces/openshift-controller-manager/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794620013Z namespaces/openshift-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794676195Z namespaces/openshift-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794739817Z namespaces/openshift-controller-manager/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794777398Z namespaces/openshift-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79485972Z namespaces/openshift-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.794898241Z namespaces/openshift-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795006684Z namespaces/openshift-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795088367Z namespaces/openshift-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795297182Z namespaces/openshift-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795336264Z namespaces/openshift-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795386185Z namespaces/openshift-controller-manager/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795452087Z namespaces/openshift-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795483748Z namespaces/openshift-controller-manager/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795491808Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79554417Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795682294Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.795883119Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796026754Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796220009Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796402154Z namespaces/openshift-controller-manager/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796448655Z namespaces/openshift-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796508597Z namespaces/openshift-controller-manager/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796551778Z namespaces/openshift-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796628191Z namespaces/openshift-dns-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796677172Z namespaces/openshift-dns-operator/openshift-dns-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796729703Z namespaces/openshift-dns-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796778365Z namespaces/openshift-dns-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.796888378Z namespaces/openshift-dns-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79695012Z namespaces/openshift-dns-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797037892Z namespaces/openshift-dns-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797113145Z namespaces/openshift-dns-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797207007Z namespaces/openshift-dns-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797264899Z namespaces/openshift-dns-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79729954Z namespaces/openshift-dns-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797376252Z namespaces/openshift-dns-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797408703Z namespaces/openshift-dns-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797498866Z namespaces/openshift-dns-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797560938Z namespaces/openshift-dns-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797609889Z namespaces/openshift-dns-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797687701Z namespaces/openshift-dns-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797766123Z namespaces/openshift-dns-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797810745Z namespaces/openshift-dns-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.797949369Z namespaces/openshift-dns-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798023821Z namespaces/openshift-dns-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798107103Z namespaces/openshift-dns-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798193026Z namespaces/openshift-dns-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798265178Z namespaces/openshift-dns-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798392651Z namespaces/openshift-dns-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798580027Z namespaces/openshift-dns-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798632518Z namespaces/openshift-dns-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79868025Z namespaces/openshift-dns-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798726181Z namespaces/openshift-dns-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798765092Z namespaces/openshift-dns-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798841434Z namespaces/openshift-dns-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798860695Z namespaces/openshift-dns-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.798948857Z namespaces/openshift-dns-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79903127Z namespaces/openshift-dns-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799067821Z namespaces/openshift-dns-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799114102Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799153183Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799227885Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/dns.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799312868Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.79937569Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/dns-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799456152Z namespaces/openshift-dns-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799497723Z namespaces/openshift-dns-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799545624Z namespaces/openshift-dns-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799556555Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799609806Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.799772271Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800003668Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800141272Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800335587Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800518362Z namespaces/openshift-dns-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800617905Z namespaces/openshift-dns-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800666247Z namespaces/openshift-dns-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800707038Z namespaces/openshift-dns-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800807391Z namespaces/openshift-dns/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800885253Z namespaces/openshift-dns/openshift-dns.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.800979405Z namespaces/openshift-dns/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.801023097Z namespaces/openshift-dns/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.80149255Z namespaces/openshift-dns/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.801518701Z namespaces/openshift-dns/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.801679896Z namespaces/openshift-dns/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.801756178Z namespaces/openshift-dns/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.801859391Z namespaces/openshift-dns/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.801919643Z namespaces/openshift-dns/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.801956933Z namespaces/openshift-dns/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802024555Z namespaces/openshift-dns/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802069337Z namespaces/openshift-dns/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802156249Z namespaces/openshift-dns/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802209091Z namespaces/openshift-dns/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802253262Z namespaces/openshift-dns/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802332904Z namespaces/openshift-dns/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802403456Z namespaces/openshift-dns/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802438277Z namespaces/openshift-dns/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802551061Z namespaces/openshift-dns/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802634123Z namespaces/openshift-dns/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.80288438Z namespaces/openshift-dns/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.802960792Z namespaces/openshift-dns/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.80322316Z namespaces/openshift-dns/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803369604Z namespaces/openshift-dns/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803552269Z namespaces/openshift-dns/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803613101Z namespaces/openshift-dns/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803658202Z namespaces/openshift-dns/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803711684Z namespaces/openshift-dns/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803759975Z namespaces/openshift-dns/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803860418Z namespaces/openshift-dns/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.803901109Z namespaces/openshift-dns/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804000652Z namespaces/openshift-dns/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804079584Z namespaces/openshift-dns/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804140066Z namespaces/openshift-dns/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804182857Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804245959Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.80429064Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/dns-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804346652Z namespaces/openshift-dns/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804389583Z namespaces/openshift-dns/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804420554Z namespaces/openshift-dns/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804431004Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804471156Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8046201Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804809315Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.804946219Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805150125Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805394402Z namespaces/openshift-dns/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805403542Z namespaces/openshift-dns/pods/dns-default-6gf8v/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805438413Z namespaces/openshift-dns/pods/dns-default-6gf8v/dns-default-6gf8v.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805533846Z namespaces/openshift-dns/pods/dns-default-6gf8v/dns/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805543056Z namespaces/openshift-dns/pods/dns-default-6gf8v/dns/dns/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805549116Z namespaces/openshift-dns/pods/dns-default-6gf8v/dns/dns/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805594218Z namespaces/openshift-dns/pods/dns-default-6gf8v/dns/dns/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805696721Z namespaces/openshift-dns/pods/dns-default-6gf8v/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805768113Z namespaces/openshift-dns/pods/dns-default-6gf8v/dns/dns/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805820854Z namespaces/openshift-dns/pods/dns-default-6gf8v/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805839925Z namespaces/openshift-dns/pods/dns-default-6gf8v/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805844215Z namespaces/openshift-dns/pods/dns-default-6gf8v/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805890486Z namespaces/openshift-dns/pods/dns-default-6gf8v/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.805991689Z namespaces/openshift-dns/pods/dns-default-6gf8v/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806105692Z namespaces/openshift-dns/pods/dns-default-6gf8v/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806143824Z namespaces/openshift-dns/pods/dns-default-6gphb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806179094Z namespaces/openshift-dns/pods/dns-default-6gphb/dns-default-6gphb.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806271777Z namespaces/openshift-dns/pods/dns-default-6gphb/dns/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806288858Z namespaces/openshift-dns/pods/dns-default-6gphb/dns/dns/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806317689Z namespaces/openshift-dns/pods/dns-default-6gphb/dns/dns/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806343189Z namespaces/openshift-dns/pods/dns-default-6gphb/dns/dns/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806420461Z namespaces/openshift-dns/pods/dns-default-6gphb/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806487154Z namespaces/openshift-dns/pods/dns-default-6gphb/dns/dns/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806523884Z namespaces/openshift-dns/pods/dns-default-6gphb/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806529835Z namespaces/openshift-dns/pods/dns-default-6gphb/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806544235Z namespaces/openshift-dns/pods/dns-default-6gphb/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806611357Z namespaces/openshift-dns/pods/dns-default-6gphb/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.80670489Z namespaces/openshift-dns/pods/dns-default-6gphb/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806771762Z namespaces/openshift-dns/pods/dns-default-6gphb/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806803103Z namespaces/openshift-dns/pods/dns-default-gxkfc/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.806908386Z namespaces/openshift-dns/pods/dns-default-gxkfc/dns-default-gxkfc.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807183063Z namespaces/openshift-dns/pods/dns-default-gxkfc/dns/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807198014Z namespaces/openshift-dns/pods/dns-default-gxkfc/dns/dns/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807205794Z namespaces/openshift-dns/pods/dns-default-gxkfc/dns/dns/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807219404Z namespaces/openshift-dns/pods/dns-default-gxkfc/dns/dns/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807351398Z namespaces/openshift-dns/pods/dns-default-gxkfc/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.80742181Z namespaces/openshift-dns/pods/dns-default-gxkfc/dns/dns/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807444141Z namespaces/openshift-dns/pods/dns-default-gxkfc/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807454721Z namespaces/openshift-dns/pods/dns-default-gxkfc/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807465342Z namespaces/openshift-dns/pods/dns-default-gxkfc/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807521093Z namespaces/openshift-dns/pods/dns-default-gxkfc/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807635196Z namespaces/openshift-dns/pods/dns-default-gxkfc/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807699238Z namespaces/openshift-dns/pods/dns-default-gxkfc/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807735629Z namespaces/openshift-dns/pods/node-resolver-5v7r9/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807814961Z namespaces/openshift-dns/pods/node-resolver-5v7r9/node-resolver-5v7r9.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807906554Z namespaces/openshift-dns/pods/node-resolver-5v7r9/dns-node-resolver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807917014Z namespaces/openshift-dns/pods/node-resolver-5v7r9/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807921264Z namespaces/openshift-dns/pods/node-resolver-5v7r9/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.807965276Z namespaces/openshift-dns/pods/node-resolver-5v7r9/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808068289Z namespaces/openshift-dns/pods/node-resolver-5v7r9/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808172002Z namespaces/openshift-dns/pods/node-resolver-5v7r9/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808223963Z namespaces/openshift-dns/pods/node-resolver-jrqq2/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808271765Z namespaces/openshift-dns/pods/node-resolver-jrqq2/node-resolver-jrqq2.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808363917Z namespaces/openshift-dns/pods/node-resolver-jrqq2/dns-node-resolver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808374217Z namespaces/openshift-dns/pods/node-resolver-jrqq2/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808378288Z namespaces/openshift-dns/pods/node-resolver-jrqq2/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808393048Z namespaces/openshift-dns/pods/node-resolver-jrqq2/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808490141Z namespaces/openshift-dns/pods/node-resolver-jrqq2/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808567673Z namespaces/openshift-dns/pods/node-resolver-jrqq2/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808588804Z namespaces/openshift-dns/pods/node-resolver-pvj74/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808643585Z namespaces/openshift-dns/pods/node-resolver-pvj74/node-resolver-pvj74.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808713607Z namespaces/openshift-dns/pods/node-resolver-pvj74/dns-node-resolver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808722147Z namespaces/openshift-dns/pods/node-resolver-pvj74/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808725958Z namespaces/openshift-dns/pods/node-resolver-pvj74/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808778009Z namespaces/openshift-dns/pods/node-resolver-pvj74/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808875092Z namespaces/openshift-dns/pods/node-resolver-pvj74/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808943914Z namespaces/openshift-dns/pods/node-resolver-pvj74/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.808980805Z namespaces/openshift-dns/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809022136Z namespaces/openshift-dns/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809068708Z namespaces/openshift-dns/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809111629Z namespaces/openshift-dns/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809196381Z namespaces/openshift-etcd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809238612Z namespaces/openshift-etcd/openshift-etcd.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809303524Z namespaces/openshift-etcd/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809345085Z namespaces/openshift-etcd/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809394247Z namespaces/openshift-etcd/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809417258Z namespaces/openshift-etcd/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809536981Z namespaces/openshift-etcd/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809622723Z namespaces/openshift-etcd/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809707406Z namespaces/openshift-etcd/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809723036Z namespaces/openshift-etcd/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.80984819Z namespaces/openshift-etcd/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809911542Z namespaces/openshift-etcd/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.809967543Z namespaces/openshift-etcd/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810069396Z namespaces/openshift-etcd/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810134258Z namespaces/openshift-etcd/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810176649Z namespaces/openshift-etcd/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810283122Z namespaces/openshift-etcd/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810360194Z namespaces/openshift-etcd/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810416326Z namespaces/openshift-etcd/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810526969Z namespaces/openshift-etcd/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810600221Z namespaces/openshift-etcd/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810684874Z namespaces/openshift-etcd/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810763536Z namespaces/openshift-etcd/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.810953331Z namespaces/openshift-etcd/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811082015Z namespaces/openshift-etcd/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.81124928Z namespaces/openshift-etcd/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811316212Z namespaces/openshift-etcd/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811367043Z namespaces/openshift-etcd/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811435365Z namespaces/openshift-etcd/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811479226Z namespaces/openshift-etcd/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811537268Z namespaces/openshift-etcd/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811571169Z namespaces/openshift-etcd/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811659612Z namespaces/openshift-etcd/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811737854Z namespaces/openshift-etcd/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811801236Z namespaces/openshift-etcd/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811866598Z namespaces/openshift-etcd/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811887908Z namespaces/openshift-etcd/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.811962011Z namespaces/openshift-etcd/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812005562Z namespaces/openshift-etcd/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812030112Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812093644Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812252899Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812429784Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812569158Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812757273Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.812975549Z namespaces/openshift-etcd/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.81301199Z namespaces/openshift-etcd/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813063432Z namespaces/openshift-etcd/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813104703Z namespaces/openshift-etcd/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813195976Z namespaces/openshift-host-network/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813239677Z namespaces/openshift-host-network/openshift-host-network.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813310089Z namespaces/openshift-host-network/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8133535Z namespaces/openshift-host-network/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813419452Z namespaces/openshift-host-network/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813462054Z namespaces/openshift-host-network/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813562046Z namespaces/openshift-host-network/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813650949Z namespaces/openshift-host-network/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813729751Z namespaces/openshift-host-network/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813771862Z namespaces/openshift-host-network/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813838404Z namespaces/openshift-host-network/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813886786Z namespaces/openshift-host-network/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.813939847Z namespaces/openshift-host-network/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814016519Z namespaces/openshift-host-network/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814076851Z namespaces/openshift-host-network/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814123233Z namespaces/openshift-host-network/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814196395Z namespaces/openshift-host-network/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814266177Z namespaces/openshift-host-network/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814313168Z namespaces/openshift-host-network/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814435902Z namespaces/openshift-host-network/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814510924Z namespaces/openshift-host-network/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814596516Z namespaces/openshift-host-network/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814702529Z namespaces/openshift-host-network/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814764891Z namespaces/openshift-host-network/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.814937336Z namespaces/openshift-host-network/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815044889Z namespaces/openshift-host-network/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815109941Z namespaces/openshift-host-network/core/resourcequotas/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815154942Z namespaces/openshift-host-network/core/resourcequotas/host-network-namespace-quotas.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815219344Z namespaces/openshift-host-network/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815246405Z namespaces/openshift-host-network/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815326687Z namespaces/openshift-host-network/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815373448Z namespaces/openshift-host-network/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.81543043Z namespaces/openshift-host-network/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815500412Z namespaces/openshift-host-network/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815597055Z namespaces/openshift-host-network/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815666117Z namespaces/openshift-host-network/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815711028Z namespaces/openshift-host-network/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815749219Z namespaces/openshift-host-network/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.81579739Z namespaces/openshift-host-network/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815862352Z namespaces/openshift-host-network/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815930024Z namespaces/openshift-host-network/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815949595Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.815986916Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.81613569Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.816306905Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.816439549Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.816643045Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.816858351Z namespaces/openshift-host-network/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.816900632Z namespaces/openshift-host-network/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.816943453Z namespaces/openshift-host-network/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.816992365Z namespaces/openshift-host-network/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.817057097Z namespaces/openshift-image-registry/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.817103648Z namespaces/openshift-image-registry/openshift-image-registry.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.817447538Z namespaces/openshift-image-registry/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.817481329Z namespaces/openshift-image-registry/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.81752957Z namespaces/openshift-image-registry/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.817573131Z namespaces/openshift-image-registry/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.817681215Z namespaces/openshift-image-registry/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.817811858Z namespaces/openshift-image-registry/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818024334Z namespaces/openshift-image-registry/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818066186Z namespaces/openshift-image-registry/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818108017Z namespaces/openshift-image-registry/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818169718Z namespaces/openshift-image-registry/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8182136Z namespaces/openshift-image-registry/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818440876Z namespaces/openshift-image-registry/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818484927Z namespaces/openshift-image-registry/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818524368Z namespaces/openshift-image-registry/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818602791Z namespaces/openshift-image-registry/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818649052Z namespaces/openshift-image-registry/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818659272Z namespaces/openshift-image-registry/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818709764Z namespaces/openshift-image-registry/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818778966Z namespaces/openshift-image-registry/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.818852098Z namespaces/openshift-image-registry/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.819778385Z namespaces/openshift-image-registry/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.819901378Z namespaces/openshift-image-registry/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820152205Z namespaces/openshift-image-registry/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820240398Z namespaces/openshift-image-registry/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820481365Z namespaces/openshift-image-registry/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820616308Z namespaces/openshift-image-registry/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820824125Z namespaces/openshift-image-registry/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820886136Z namespaces/openshift-image-registry/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820935618Z namespaces/openshift-image-registry/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.820992159Z namespaces/openshift-image-registry/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821039351Z namespaces/openshift-image-registry/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821097112Z namespaces/openshift-image-registry/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821143924Z namespaces/openshift-image-registry/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821223816Z namespaces/openshift-image-registry/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821330709Z namespaces/openshift-image-registry/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821392541Z namespaces/openshift-image-registry/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821442842Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821516225Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821563106Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-operator-alerts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821662869Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821749231Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/imagestreams-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821802563Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821869915Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/image-registry.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821924106Z namespaces/openshift-image-registry/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.821975298Z namespaces/openshift-image-registry/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.822003659Z namespaces/openshift-image-registry/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8220479Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.822089141Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.822268386Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.822487492Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.822725819Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.822940445Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.823128041Z namespaces/openshift-image-registry/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.823136991Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.823170562Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/image-registry-676884554-ss85x.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.823278145Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/registry/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.823289725Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/registry/registry/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.823293795Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/registry/registry/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.823345987Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/registry/registry/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825058816Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/registry/registry/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825123888Z namespaces/openshift-image-registry/pods/image-registry-676884554-ss85x/registry/registry/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825155049Z namespaces/openshift-image-registry/pods/node-ca-7j26t/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8251949Z namespaces/openshift-image-registry/pods/node-ca-7j26t/node-ca-7j26t.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825273462Z namespaces/openshift-image-registry/pods/node-ca-7j26t/node-ca/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825281793Z namespaces/openshift-image-registry/pods/node-ca-7j26t/node-ca/node-ca/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825286202Z namespaces/openshift-image-registry/pods/node-ca-7j26t/node-ca/node-ca/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825309543Z namespaces/openshift-image-registry/pods/node-ca-7j26t/node-ca/node-ca/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825467558Z namespaces/openshift-image-registry/pods/node-ca-7j26t/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.82553254Z namespaces/openshift-image-registry/pods/node-ca-7j26t/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825577371Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825608712Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/node-ca-ndrg7.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825696904Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/node-ca/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825705225Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/node-ca/node-ca/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825709225Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/node-ca/node-ca/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825740636Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/node-ca/node-ca/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.825926781Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826003373Z namespaces/openshift-image-registry/pods/node-ca-ndrg7/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826022304Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826065535Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/node-ca-zp5mq.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826150737Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/node-ca/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826162278Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/node-ca/node-ca/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826166958Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/node-ca/node-ca/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826219659Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/node-ca/node-ca/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826375014Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826443166Z namespaces/openshift-image-registry/pods/node-ca-zp5mq/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826460996Z namespaces/openshift-image-registry/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826508217Z namespaces/openshift-image-registry/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826568879Z namespaces/openshift-image-registry/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.82660408Z namespaces/openshift-image-registry/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826657312Z namespaces/openshift-infra/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826666142Z namespaces/openshift-infra/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826671212Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826682122Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.826891939Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827064833Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827201288Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827405903Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.82764727Z namespaces/openshift-ingress-canary/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827695332Z namespaces/openshift-ingress-canary/openshift-ingress-canary.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827770284Z namespaces/openshift-ingress-canary/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827835106Z namespaces/openshift-ingress-canary/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827898477Z namespaces/openshift-ingress-canary/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.827947789Z namespaces/openshift-ingress-canary/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828053112Z namespaces/openshift-ingress-canary/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828128894Z namespaces/openshift-ingress-canary/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828213617Z namespaces/openshift-ingress-canary/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828269968Z namespaces/openshift-ingress-canary/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828319309Z namespaces/openshift-ingress-canary/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828459224Z namespaces/openshift-ingress-canary/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828479534Z namespaces/openshift-ingress-canary/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828573767Z namespaces/openshift-ingress-canary/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828623288Z namespaces/openshift-ingress-canary/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.82867495Z namespaces/openshift-ingress-canary/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828743882Z namespaces/openshift-ingress-canary/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828840455Z namespaces/openshift-ingress-canary/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.828893216Z namespaces/openshift-ingress-canary/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829009009Z namespaces/openshift-ingress-canary/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829098072Z namespaces/openshift-ingress-canary/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829244086Z namespaces/openshift-ingress-canary/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829319748Z namespaces/openshift-ingress-canary/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829486823Z namespaces/openshift-ingress-canary/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829603556Z namespaces/openshift-ingress-canary/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829765811Z namespaces/openshift-ingress-canary/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829851753Z namespaces/openshift-ingress-canary/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829889305Z namespaces/openshift-ingress-canary/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.829963427Z namespaces/openshift-ingress-canary/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830006888Z namespaces/openshift-ingress-canary/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830048229Z namespaces/openshift-ingress-canary/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83008879Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830176513Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830254805Z namespaces/openshift-ingress-canary/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830311597Z namespaces/openshift-ingress-canary/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830355578Z namespaces/openshift-ingress-canary/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830412Z namespaces/openshift-ingress-canary/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830467331Z namespaces/openshift-ingress-canary/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830532893Z namespaces/openshift-ingress-canary/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830544673Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830596425Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83076389Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.830983136Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83111459Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831326936Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831770178Z namespaces/openshift-ingress-canary/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831779099Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831849361Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/ingress-canary-mbhc7.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831932213Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/serve-healthcheck-canary/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831940873Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831954754Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.831997595Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832101808Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83216369Z namespaces/openshift-ingress-canary/pods/ingress-canary-mbhc7/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832209271Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832255142Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/ingress-canary-wfcbh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832327735Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/serve-healthcheck-canary/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832338585Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832352555Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832391036Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832489869Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832559931Z namespaces/openshift-ingress-canary/pods/ingress-canary-wfcbh/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832600532Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832647604Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/ingress-canary-xhkjb.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832745636Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/serve-healthcheck-canary/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832759937Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832768517Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832807528Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.832936792Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833008114Z namespaces/openshift-ingress-canary/pods/ingress-canary-xhkjb/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833049525Z namespaces/openshift-ingress-canary/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833068036Z namespaces/openshift-ingress-canary/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833138558Z namespaces/openshift-ingress-canary/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833180089Z namespaces/openshift-ingress-canary/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833305872Z namespaces/openshift-ingress-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833324663Z namespaces/openshift-ingress-operator/openshift-ingress-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833411335Z namespaces/openshift-ingress-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833444096Z namespaces/openshift-ingress-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833515898Z namespaces/openshift-ingress-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833543469Z namespaces/openshift-ingress-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833635922Z namespaces/openshift-ingress-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833711524Z namespaces/openshift-ingress-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833809697Z namespaces/openshift-ingress-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833872749Z namespaces/openshift-ingress-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83391991Z namespaces/openshift-ingress-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.833966891Z namespaces/openshift-ingress-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.834007352Z namespaces/openshift-ingress-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.834086845Z namespaces/openshift-ingress-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.834149037Z namespaces/openshift-ingress-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.834176637Z namespaces/openshift-ingress-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83426894Z namespaces/openshift-ingress-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.834333642Z namespaces/openshift-ingress-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.834382863Z namespaces/openshift-ingress-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.835382572Z namespaces/openshift-ingress-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.835464004Z namespaces/openshift-ingress-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.835604588Z namespaces/openshift-ingress-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83567604Z namespaces/openshift-ingress-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.835762243Z namespaces/openshift-ingress-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.835910727Z namespaces/openshift-ingress-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836093192Z namespaces/openshift-ingress-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836162034Z namespaces/openshift-ingress-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836223016Z namespaces/openshift-ingress-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836283998Z namespaces/openshift-ingress-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836321599Z namespaces/openshift-ingress-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836379651Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836389791Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836424612Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/default-wildcard.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836510875Z namespaces/openshift-ingress-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836564406Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836667959Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836767472Z namespaces/openshift-ingress-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836860745Z namespaces/openshift-ingress-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.836909606Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837003518Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83703479Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ingress-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837142493Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837198194Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ingress-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837258546Z namespaces/openshift-ingress-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837293097Z namespaces/openshift-ingress-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837340088Z namespaces/openshift-ingress-operator/operator.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837352868Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83739832Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837479042Z namespaces/openshift-ingress-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837491503Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837541674Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837702249Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.837907984Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838056519Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838272375Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83845485Z namespaces/openshift-ingress-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838494091Z namespaces/openshift-ingress-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838544883Z namespaces/openshift-ingress-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838577184Z namespaces/openshift-ingress-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838689947Z namespaces/openshift-ingress/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838737078Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.83880698Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838873662Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838948294Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.838998126Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839087598Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839335656Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839547182Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839563272Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839620033Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839681955Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839726157Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839815549Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839894492Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.839908862Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840025465Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840339504Z namespaces/openshift-ingress/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840358125Z namespaces/openshift-ingress/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840369785Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-ca-openshift-gateway.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840519959Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-deployment-openshift-gateway.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840610102Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-status-leader-openshift-gateway.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840726475Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.840766237Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.841306602Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.841403735Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.841633401Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.841701053Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.841978191Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842105545Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842325221Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842404223Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842454935Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842543227Z namespaces/openshift-ingress/gateway.networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842562718Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842618739Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/openshift-ai-inference.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842704432Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842752203Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842817185Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.842942689Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843023041Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843107364Z namespaces/openshift-ingress/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843167795Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843199816Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843285059Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.84332611Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/router-default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843382872Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843409182Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843478154Z namespaces/openshift-ingress/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843489544Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843543366Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843700201Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.843895416Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844051721Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844270297Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844472033Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844481053Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844515954Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/istiod-openshift-gateway-94bb8fbfd-fwgmd.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844642677Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844656428Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844664438Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.844694069Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.858720331Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.858806984Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-fwgmd/discovery/discovery/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.858857765Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.858916807Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.859044821Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.859062271Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.859068461Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.859097272Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.859890885Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-4jvzv/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860291016Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860334947Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router-default-5d495bcbc5-w68bb.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860476912Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860485802Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860499542Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860543513Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860656737Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860719629Z namespaces/openshift-ingress/pods/router-default-5d495bcbc5-w68bb/router/router/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86075021Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860807761Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860888354Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.860921075Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861029458Z namespaces/openshift-insights/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861071279Z namespaces/openshift-insights/openshift-insights.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861153101Z namespaces/openshift-insights/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861182282Z namespaces/openshift-insights/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861259664Z namespaces/openshift-insights/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861289955Z namespaces/openshift-insights/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861418789Z namespaces/openshift-insights/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861529662Z namespaces/openshift-insights/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861635075Z namespaces/openshift-insights/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861664396Z namespaces/openshift-insights/apps/deployments/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861715687Z namespaces/openshift-insights/apps/deployments/insights-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86180516Z namespaces/openshift-insights/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861842611Z namespaces/openshift-insights/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861911573Z namespaces/openshift-insights/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.861952924Z namespaces/openshift-insights/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.862039207Z namespaces/openshift-insights/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.862094268Z namespaces/openshift-insights/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.862125189Z namespaces/openshift-insights/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.862214641Z namespaces/openshift-insights/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.862289354Z namespaces/openshift-insights/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.862337445Z namespaces/openshift-insights/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.863190089Z namespaces/openshift-insights/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.863296023Z namespaces/openshift-insights/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.863667773Z namespaces/openshift-insights/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.863804207Z namespaces/openshift-insights/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864095526Z namespaces/openshift-insights/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864184448Z namespaces/openshift-insights/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864340522Z namespaces/openshift-insights/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864402954Z namespaces/openshift-insights/core/configmaps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864445676Z namespaces/openshift-insights/core/configmaps/service-ca-bundle.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864510247Z namespaces/openshift-insights/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864554089Z namespaces/openshift-insights/core/serviceaccounts/gather.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864639701Z namespaces/openshift-insights/core/serviceaccounts/operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864702593Z namespaces/openshift-insights/core/services/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864742004Z namespaces/openshift-insights/core/services/metrics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864806016Z namespaces/openshift-insights/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864869008Z namespaces/openshift-insights/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86493287Z namespaces/openshift-insights/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.864975201Z namespaces/openshift-insights/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865032802Z namespaces/openshift-insights/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865083824Z namespaces/openshift-insights/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865160016Z namespaces/openshift-insights/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865233518Z namespaces/openshift-insights/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86528775Z namespaces/openshift-insights/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865323231Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865389743Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865433394Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/insights-prometheus-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865520796Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865540927Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/insights-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865624459Z namespaces/openshift-insights/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86564367Z namespaces/openshift-insights/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865702712Z namespaces/openshift-insights/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865714082Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.865759883Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86601317Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866220786Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86636602Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866568626Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866818564Z namespaces/openshift-insights/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866844764Z namespaces/openshift-insights/pods/insights-operator-5c847dcfc4-n9nrj/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866861445Z namespaces/openshift-insights/pods/insights-operator-5c847dcfc4-n9nrj/insights-operator-5c847dcfc4-n9nrj.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866967108Z namespaces/openshift-insights/pods/insights-operator-5c847dcfc4-n9nrj/insights-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866976388Z namespaces/openshift-insights/pods/insights-operator-5c847dcfc4-n9nrj/insights-operator/insights-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.866990098Z namespaces/openshift-insights/pods/insights-operator-5c847dcfc4-n9nrj/insights-operator/insights-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86704448Z namespaces/openshift-insights/pods/insights-operator-5c847dcfc4-n9nrj/insights-operator/insights-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.867441071Z namespaces/openshift-insights/pods/insights-operator-5c847dcfc4-n9nrj/insights-operator/insights-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86773429Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86775362Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/insights-runtime-extractor-92n4w.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.867921755Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.867937106Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/exporter/exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.867942266Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/exporter/exporter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.867964926Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/exporter/exporter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868056209Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868122811Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/exporter/exporter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868146122Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/extractor/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868158512Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/extractor/extractor/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868168632Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/extractor/extractor/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868203623Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/extractor/extractor/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868306786Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868368838Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/extractor/extractor/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868396039Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868412409Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86842713Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868469671Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868583084Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868661486Z namespaces/openshift-insights/pods/insights-runtime-extractor-92n4w/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868695827Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868750309Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/insights-runtime-extractor-dtzt4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868871152Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868887253Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/exporter/exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868898253Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/exporter/exporter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.868932074Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/exporter/exporter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869026367Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869100919Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/exporter/exporter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869117959Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/extractor/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.86913635Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/extractor/extractor/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869149711Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/extractor/extractor/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869194882Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/extractor/extractor/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869286724Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869362206Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/extractor/extractor/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869398327Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869405138Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869412858Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869424288Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869537691Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869604684Z namespaces/openshift-insights/pods/insights-runtime-extractor-dtzt4/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869783418Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8698452Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/insights-runtime-extractor-f2vbk.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869942333Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869952433Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/exporter/exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869956794Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/exporter/exporter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.869982844Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/exporter/exporter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870075487Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870142749Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/exporter/exporter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87019294Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/extractor/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870205551Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/extractor/extractor/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870211871Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/extractor/extractor/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870222001Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/extractor/extractor/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870321844Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870409876Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/extractor/extractor/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870425957Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870433107Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870440207Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870495289Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870598112Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870670734Z namespaces/openshift-insights/pods/insights-runtime-extractor-f2vbk/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870696095Z namespaces/openshift-insights/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870743876Z namespaces/openshift-insights/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870810108Z namespaces/openshift-insights/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870831959Z namespaces/openshift-insights/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870902681Z namespaces/openshift-keda/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870913581Z namespaces/openshift-keda/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870919021Z namespaces/openshift-keda/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.870949742Z namespaces/openshift-keda/coordination.k8s.io/leases/olm-operator.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871034644Z namespaces/openshift-keda/coordination.k8s.io/leases/operator.keda.sh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871076946Z namespaces/openshift-keda/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871085496Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871118647Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/keda-olm-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871172118Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871212259Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-admission-webhooks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871317682Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-metrics-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871400885Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871446856Z namespaces/openshift-keda/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871458037Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871529389Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871662272Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.871864578Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/custom-metrics-autoscaler.v2.19.0-1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872058074Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87226869Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872387593Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872558048Z namespaces/openshift-keda/operators.coreos.com/installplans/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872592139Z namespaces/openshift-keda/operators.coreos.com/installplans/install-qqcnv.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872706702Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872728753Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/custom-metrics-autoscaler.v2.19.0-1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872809915Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872854006Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/openshift-keda.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872912708Z namespaces/openshift-keda/operators.coreos.com/subscriptions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.872937839Z namespaces/openshift-keda/operators.coreos.com/subscriptions/openshift-custom-metrics-autoscaler-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873035492Z namespaces/openshift-kube-apiserver-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873147935Z namespaces/openshift-kube-apiserver-operator/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873207837Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873255268Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873306039Z namespaces/openshift-kube-apiserver-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873358611Z namespaces/openshift-kube-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873434023Z namespaces/openshift-kube-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873522316Z namespaces/openshift-kube-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873593018Z namespaces/openshift-kube-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873635739Z namespaces/openshift-kube-apiserver-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87366645Z namespaces/openshift-kube-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873725292Z namespaces/openshift-kube-apiserver-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873755903Z namespaces/openshift-kube-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873874376Z namespaces/openshift-kube-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873931068Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.873953648Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874043721Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874112243Z namespaces/openshift-kube-apiserver-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874311908Z namespaces/openshift-kube-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874320659Z namespaces/openshift-kube-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874327099Z namespaces/openshift-kube-apiserver-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874425702Z namespaces/openshift-kube-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874502484Z namespaces/openshift-kube-apiserver-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874580976Z namespaces/openshift-kube-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874699199Z namespaces/openshift-kube-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874895085Z namespaces/openshift-kube-apiserver-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874953297Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.874992748Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87505293Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875105431Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875162323Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875200954Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875288246Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875372139Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87540495Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875462992Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875501902Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875542964Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875571544Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875580775Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875640557Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875803741Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.875982406Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87611001Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876304585Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876489211Z namespaces/openshift-kube-apiserver-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876530642Z namespaces/openshift-kube-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876584284Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876622025Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876723528Z namespaces/openshift-kube-apiserver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876761539Z namespaces/openshift-kube-apiserver/openshift-kube-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876843401Z namespaces/openshift-kube-apiserver/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876886582Z namespaces/openshift-kube-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876951044Z namespaces/openshift-kube-apiserver/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.876989225Z namespaces/openshift-kube-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877074308Z namespaces/openshift-kube-apiserver/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87714984Z namespaces/openshift-kube-apiserver/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877230982Z namespaces/openshift-kube-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877270433Z namespaces/openshift-kube-apiserver/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877300524Z namespaces/openshift-kube-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877365156Z namespaces/openshift-kube-apiserver/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877409607Z namespaces/openshift-kube-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877486299Z namespaces/openshift-kube-apiserver/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877549211Z namespaces/openshift-kube-apiserver/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877592913Z namespaces/openshift-kube-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877671085Z namespaces/openshift-kube-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877739197Z namespaces/openshift-kube-apiserver/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877781128Z namespaces/openshift-kube-apiserver/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877921602Z namespaces/openshift-kube-apiserver/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.877976393Z namespaces/openshift-kube-apiserver/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878058886Z namespaces/openshift-kube-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878135798Z namespaces/openshift-kube-apiserver/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87821782Z namespaces/openshift-kube-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878339984Z namespaces/openshift-kube-apiserver/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878483798Z namespaces/openshift-kube-apiserver/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878524999Z namespaces/openshift-kube-apiserver/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87856448Z namespaces/openshift-kube-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878618622Z namespaces/openshift-kube-apiserver/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878659273Z namespaces/openshift-kube-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878695374Z namespaces/openshift-kube-apiserver/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878736815Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.87888924Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.878968722Z namespaces/openshift-kube-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879030544Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879068695Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879145217Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879188878Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/api-usage.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879325942Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/podsecurity.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879349493Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879415645Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/openshift-kube-apiserver.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879511348Z namespaces/openshift-kube-apiserver/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879533968Z namespaces/openshift-kube-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879628171Z namespaces/openshift-kube-apiserver/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879640701Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879650452Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.879844737Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880023842Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880159226Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880350612Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880534907Z namespaces/openshift-kube-apiserver/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880567268Z namespaces/openshift-kube-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880627789Z namespaces/openshift-kube-apiserver/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8806567Z namespaces/openshift-kube-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880731903Z namespaces/openshift-kube-controller-manager-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880769444Z namespaces/openshift-kube-controller-manager-operator/openshift-kube-controller-manager-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880853696Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880874267Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880950379Z namespaces/openshift-kube-controller-manager-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.880967709Z namespaces/openshift-kube-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881071702Z namespaces/openshift-kube-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881147524Z namespaces/openshift-kube-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881228257Z namespaces/openshift-kube-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881265688Z namespaces/openshift-kube-controller-manager-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881315289Z namespaces/openshift-kube-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88135903Z namespaces/openshift-kube-controller-manager-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881408422Z namespaces/openshift-kube-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881486424Z namespaces/openshift-kube-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881535216Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881577017Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881649189Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881709891Z namespaces/openshift-kube-controller-manager-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881754952Z namespaces/openshift-kube-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881876235Z namespaces/openshift-kube-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.881955588Z namespaces/openshift-kube-controller-manager-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88203335Z namespaces/openshift-kube-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882115512Z namespaces/openshift-kube-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882190284Z namespaces/openshift-kube-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882311248Z namespaces/openshift-kube-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882481673Z namespaces/openshift-kube-controller-manager-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882539964Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882579825Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88274471Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882781721Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882880754Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882905135Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.882994087Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88306989Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883115441Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883156692Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883197343Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883247015Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/kube-controller-manager-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883305906Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883342977Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883386159Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883396879Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88343568Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883581934Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883760009Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.883920924Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884105549Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884278034Z namespaces/openshift-kube-controller-manager-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884322526Z namespaces/openshift-kube-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884363517Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884403818Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8844737Z namespaces/openshift-kube-controller-manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884516681Z namespaces/openshift-kube-controller-manager/openshift-kube-controller-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884575133Z namespaces/openshift-kube-controller-manager/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884610394Z namespaces/openshift-kube-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884652425Z namespaces/openshift-kube-controller-manager/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884718567Z namespaces/openshift-kube-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884783119Z namespaces/openshift-kube-controller-manager/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884891332Z namespaces/openshift-kube-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.884969404Z namespaces/openshift-kube-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885030396Z namespaces/openshift-kube-controller-manager/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885049216Z namespaces/openshift-kube-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885117898Z namespaces/openshift-kube-controller-manager/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885137219Z namespaces/openshift-kube-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885236912Z namespaces/openshift-kube-controller-manager/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885256232Z namespaces/openshift-kube-controller-manager/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885326444Z namespaces/openshift-kube-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885396056Z namespaces/openshift-kube-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885439287Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885447238Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885485689Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/cluster-policy-controller-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885562561Z namespaces/openshift-kube-controller-manager/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885599242Z namespaces/openshift-kube-controller-manager/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885698005Z namespaces/openshift-kube-controller-manager/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.885773047Z namespaces/openshift-kube-controller-manager/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886069796Z namespaces/openshift-kube-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886149738Z namespaces/openshift-kube-controller-manager/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88622812Z namespaces/openshift-kube-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886350284Z namespaces/openshift-kube-controller-manager/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886512188Z namespaces/openshift-kube-controller-manager/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886554599Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886602171Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886654402Z namespaces/openshift-kube-controller-manager/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886688153Z namespaces/openshift-kube-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886748905Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886802027Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.886989322Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887068994Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887109345Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887155517Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887203948Z namespaces/openshift-kube-controller-manager/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887241999Z namespaces/openshift-kube-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887292321Z namespaces/openshift-kube-controller-manager/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887302261Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887338692Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887487306Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887666731Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.887819526Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888089114Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888266899Z namespaces/openshift-kube-controller-manager/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88830409Z namespaces/openshift-kube-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888343421Z namespaces/openshift-kube-controller-manager/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888400392Z namespaces/openshift-kube-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888438404Z namespaces/openshift-kube-scheduler-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888503545Z namespaces/openshift-kube-scheduler-operator/openshift-kube-scheduler-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888559997Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888589078Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88865282Z namespaces/openshift-kube-scheduler-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888683391Z namespaces/openshift-kube-scheduler-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888772693Z namespaces/openshift-kube-scheduler-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888875186Z namespaces/openshift-kube-scheduler-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888957788Z namespaces/openshift-kube-scheduler-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.888990929Z namespaces/openshift-kube-scheduler-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889040261Z namespaces/openshift-kube-scheduler-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889082072Z namespaces/openshift-kube-scheduler-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889113023Z namespaces/openshift-kube-scheduler-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889213006Z namespaces/openshift-kube-scheduler-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889229816Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889287148Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88936759Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889419622Z namespaces/openshift-kube-scheduler-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889447122Z namespaces/openshift-kube-scheduler-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889560396Z namespaces/openshift-kube-scheduler-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889635488Z namespaces/openshift-kube-scheduler-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.88972065Z namespaces/openshift-kube-scheduler-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889807083Z namespaces/openshift-kube-scheduler-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.889893525Z namespaces/openshift-kube-scheduler-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890018679Z namespaces/openshift-kube-scheduler-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890200374Z namespaces/openshift-kube-scheduler-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890234705Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890287237Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890343148Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890376329Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890431361Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890467382Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890552064Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890630576Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890665897Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890696798Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89075268Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890783221Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890858653Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890870923Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.890904294Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891055388Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891239514Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891385418Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891584284Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891763829Z namespaces/openshift-kube-scheduler-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89181724Z namespaces/openshift-kube-scheduler-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891891252Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891911873Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.891989825Z namespaces/openshift-kube-scheduler/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892023896Z namespaces/openshift-kube-scheduler/openshift-kube-scheduler.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892087868Z namespaces/openshift-kube-scheduler/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892119789Z namespaces/openshift-kube-scheduler/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89215868Z namespaces/openshift-kube-scheduler/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892202432Z namespaces/openshift-kube-scheduler/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892284574Z namespaces/openshift-kube-scheduler/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892367466Z namespaces/openshift-kube-scheduler/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892442628Z namespaces/openshift-kube-scheduler/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892478679Z namespaces/openshift-kube-scheduler/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892530151Z namespaces/openshift-kube-scheduler/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892572492Z namespaces/openshift-kube-scheduler/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892609833Z namespaces/openshift-kube-scheduler/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892699316Z namespaces/openshift-kube-scheduler/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892730257Z namespaces/openshift-kube-scheduler/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892767768Z namespaces/openshift-kube-scheduler/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892877281Z namespaces/openshift-kube-scheduler/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892924632Z namespaces/openshift-kube-scheduler/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.892975224Z namespaces/openshift-kube-scheduler/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893073566Z namespaces/openshift-kube-scheduler/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893150879Z namespaces/openshift-kube-scheduler/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893233231Z namespaces/openshift-kube-scheduler/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893310403Z namespaces/openshift-kube-scheduler/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893393146Z namespaces/openshift-kube-scheduler/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893524359Z namespaces/openshift-kube-scheduler/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893704755Z namespaces/openshift-kube-scheduler/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893750176Z namespaces/openshift-kube-scheduler/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893812318Z namespaces/openshift-kube-scheduler/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893860539Z namespaces/openshift-kube-scheduler/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89390738Z namespaces/openshift-kube-scheduler/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.893958412Z namespaces/openshift-kube-scheduler/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894016313Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894075525Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894162538Z namespaces/openshift-kube-scheduler/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894194169Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8942461Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894284351Z namespaces/openshift-kube-scheduler/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894329893Z namespaces/openshift-kube-scheduler/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894386724Z namespaces/openshift-kube-scheduler/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894407615Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894418965Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8945729Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.894809866Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.8949537Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895150526Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895350422Z namespaces/openshift-kube-scheduler/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895390433Z namespaces/openshift-kube-scheduler/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895429214Z namespaces/openshift-kube-scheduler/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895464115Z namespaces/openshift-kube-scheduler/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895548717Z namespaces/openshift-kube-storage-version-migrator-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895598349Z namespaces/openshift-kube-storage-version-migrator-operator/openshift-kube-storage-version-migrator-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895662831Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895704302Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895762883Z namespaces/openshift-kube-storage-version-migrator-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895800075Z namespaces/openshift-kube-storage-version-migrator-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.895944009Z namespaces/openshift-kube-storage-version-migrator-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896056442Z namespaces/openshift-kube-storage-version-migrator-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896154395Z namespaces/openshift-kube-storage-version-migrator-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896208706Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896259428Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896316429Z namespaces/openshift-kube-storage-version-migrator-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896359581Z namespaces/openshift-kube-storage-version-migrator-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896438553Z namespaces/openshift-kube-storage-version-migrator-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896494004Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896538336Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896617428Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896658179Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89666926Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896723311Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/openshift-kube-storage-version-migrator-operator-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896843845Z namespaces/openshift-kube-storage-version-migrator-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896884796Z namespaces/openshift-kube-storage-version-migrator-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.896994439Z namespaces/openshift-kube-storage-version-migrator-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897077951Z namespaces/openshift-kube-storage-version-migrator-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897254236Z namespaces/openshift-kube-storage-version-migrator-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897333268Z namespaces/openshift-kube-storage-version-migrator-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897456902Z namespaces/openshift-kube-storage-version-migrator-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897582516Z namespaces/openshift-kube-storage-version-migrator-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897757941Z namespaces/openshift-kube-storage-version-migrator-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897832733Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897886604Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897929186Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.897980527Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898028339Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89807728Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898162012Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898236785Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898296176Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898338827Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898397069Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89843395Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898498462Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898508092Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898554203Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898700438Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.898897013Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899029937Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899221323Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899433109Z namespaces/openshift-kube-storage-version-migrator-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899444989Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-846bf668f7-25p9j/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89948349Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-846bf668f7-25p9j/kube-storage-version-migrator-operator-846bf668f7-25p9j.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899607074Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-846bf668f7-25p9j/kube-storage-version-migrator-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899616224Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-846bf668f7-25p9j/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899629234Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-846bf668f7-25p9j/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899684246Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-846bf668f7-25p9j/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.89980993Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-846bf668f7-25p9j/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899948744Z namespaces/openshift-kube-storage-version-migrator-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.899969434Z namespaces/openshift-kube-storage-version-migrator-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900030296Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900054177Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900131509Z namespaces/openshift-kube-storage-version-migrator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900151379Z namespaces/openshift-kube-storage-version-migrator/openshift-kube-storage-version-migrator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900254802Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900276483Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900345845Z namespaces/openshift-kube-storage-version-migrator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900365855Z namespaces/openshift-kube-storage-version-migrator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900567811Z namespaces/openshift-kube-storage-version-migrator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900678364Z namespaces/openshift-kube-storage-version-migrator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900772957Z namespaces/openshift-kube-storage-version-migrator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900838829Z namespaces/openshift-kube-storage-version-migrator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90087018Z namespaces/openshift-kube-storage-version-migrator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900933522Z namespaces/openshift-kube-storage-version-migrator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.900979853Z namespaces/openshift-kube-storage-version-migrator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901050605Z namespaces/openshift-kube-storage-version-migrator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901095867Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901126377Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90121951Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901256001Z namespaces/openshift-kube-storage-version-migrator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901286542Z namespaces/openshift-kube-storage-version-migrator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901393495Z namespaces/openshift-kube-storage-version-migrator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901469607Z namespaces/openshift-kube-storage-version-migrator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901589981Z namespaces/openshift-kube-storage-version-migrator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901666033Z namespaces/openshift-kube-storage-version-migrator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901781746Z namespaces/openshift-kube-storage-version-migrator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.901929851Z namespaces/openshift-kube-storage-version-migrator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902098875Z namespaces/openshift-kube-storage-version-migrator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902137656Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902197848Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902250289Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90226973Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902337212Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902370993Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902455786Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902529528Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902573599Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90261324Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902670912Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902710573Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902754074Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902767604Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.902911468Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903061763Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903235548Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903386062Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903569287Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903745223Z namespaces/openshift-kube-storage-version-migrator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903753593Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903803524Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/migrator-d57d58cc9-wk5mk.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903894667Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/graceful-termination/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903902927Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/graceful-termination/graceful-termination/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903915047Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/graceful-termination/graceful-termination/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.903956318Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/graceful-termination/graceful-termination/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904043711Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/graceful-termination/graceful-termination/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904106543Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/graceful-termination/graceful-termination/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904133224Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/migrator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904138434Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/migrator/migrator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904142354Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/migrator/migrator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904181415Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/migrator/migrator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904286318Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/migrator/migrator/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90434994Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-d57d58cc9-wk5mk/migrator/migrator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904387211Z namespaces/openshift-kube-storage-version-migrator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904671419Z namespaces/openshift-kube-storage-version-migrator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90470876Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90471651Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9047259Z namespaces/openshift-lws-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904731441Z namespaces/openshift-lws-operator/openshift-lws-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904736431Z namespaces/openshift-lws-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904753101Z namespaces/openshift-lws-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904761201Z namespaces/openshift-lws-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904872505Z namespaces/openshift-lws-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.904977478Z namespaces/openshift-lws-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.905167893Z namespaces/openshift-lws-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.905300597Z namespaces/openshift-lws-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.905371559Z namespaces/openshift-lws-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.905504763Z namespaces/openshift-lws-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90609356Z namespaces/openshift-lws-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90610809Z namespaces/openshift-lws-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90611242Z namespaces/openshift-lws-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906116951Z namespaces/openshift-lws-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906120811Z namespaces/openshift-lws-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906124821Z namespaces/openshift-lws-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906128521Z namespaces/openshift-lws-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906131961Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906135361Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/b8b2488c.x-k8s.io.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906139001Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/openshift-lws-operator-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906143381Z namespaces/openshift-lws-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906149721Z namespaces/openshift-lws-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906226064Z namespaces/openshift-lws-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906308366Z namespaces/openshift-lws-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906610165Z namespaces/openshift-lws-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.906681377Z namespaces/openshift-lws-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907122889Z namespaces/openshift-lws-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907246593Z namespaces/openshift-lws-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907441749Z namespaces/openshift-lws-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90749944Z namespaces/openshift-lws-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907541051Z namespaces/openshift-lws-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907604933Z namespaces/openshift-lws-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907644934Z namespaces/openshift-lws-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907693666Z namespaces/openshift-lws-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907726607Z namespaces/openshift-lws-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90783283Z namespaces/openshift-lws-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907912382Z namespaces/openshift-lws-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.907953063Z namespaces/openshift-lws-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908003625Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908059166Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908110438Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/lws-controller-manager-metrics-monitor.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908162149Z namespaces/openshift-lws-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908213641Z namespaces/openshift-lws-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908268052Z namespaces/openshift-lws-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908276413Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908326294Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908470988Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908653423Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.908807278Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909033294Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90923652Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909429926Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909477247Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/install-7lp5c.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909551839Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.90959011Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909642981Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909679242Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/leader-worker-set.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909732464Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909776575Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/leader-worker-set.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909855838Z namespaces/openshift-lws-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909869658Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909905439Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/lws-controller-manager-74b8545fc-bpt86.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909991322Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.909998592Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910003582Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910046253Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910148276Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910217698Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-bpt86/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910263119Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91029903Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/lws-controller-manager-74b8545fc-ksrpv.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910381733Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910392793Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/manager/manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910397583Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/manager/manager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910431404Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/manager/manager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91064253Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910707702Z namespaces/openshift-lws-operator/pods/lws-controller-manager-74b8545fc-ksrpv/manager/manager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910753693Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.910809915Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/openshift-lws-operator-fd8ccff4c-d22pr.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91099345Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/openshift-lws-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911002071Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/openshift-lws-operator/openshift-lws-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911015161Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/openshift-lws-operator/openshift-lws-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911048112Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/openshift-lws-operator/openshift-lws-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911208646Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/openshift-lws-operator/openshift-lws-operator/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911288769Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-d22pr/openshift-lws-operator/openshift-lws-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91131988Z namespaces/openshift-lws-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911367361Z namespaces/openshift-lws-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911405102Z namespaces/openshift-lws-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911451764Z namespaces/openshift-lws-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911485844Z namespaces/openshift-machine-api/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911492474Z namespaces/openshift-machine-api/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911504375Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911542076Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91169782Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.911977648Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912116492Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912344679Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912522154Z namespaces/openshift-machine-config-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912531395Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912535624Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912557035Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-controller.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912665078Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-daemon.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91273738Z namespaces/openshift-machine-config-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91274527Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912783932Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.912960317Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913135922Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913266125Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913500482Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913684577Z namespaces/openshift-marketplace/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913696718Z namespaces/openshift-marketplace/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913701658Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913721718Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/marketplace-alert-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913808771Z namespaces/openshift-marketplace/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913820371Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913858213Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/certified-operators.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.913949375Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/community-operators.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.914028617Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-marketplace.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91411065Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-operators.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.914171231Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.914205583Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.914351247Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.914531832Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.914665646Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.914876732Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.915076077Z namespaces/openshift-monitoring/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.915111718Z namespaces/openshift-monitoring/openshift-monitoring.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.915203381Z namespaces/openshift-monitoring/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.915260963Z namespaces/openshift-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.915343235Z namespaces/openshift-monitoring/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.915378326Z namespaces/openshift-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91552046Z namespaces/openshift-monitoring/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.915946952Z namespaces/openshift-monitoring/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916286402Z namespaces/openshift-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916513679Z namespaces/openshift-monitoring/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.91655304Z namespaces/openshift-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916599371Z namespaces/openshift-monitoring/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916642292Z namespaces/openshift-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916728425Z namespaces/openshift-monitoring/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916813877Z namespaces/openshift-monitoring/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916868809Z namespaces/openshift-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916949751Z namespaces/openshift-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.916998772Z namespaces/openshift-monitoring/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.917035073Z namespaces/openshift-monitoring/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.920242336Z namespaces/openshift-monitoring/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92037916Z namespaces/openshift-monitoring/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.921269895Z namespaces/openshift-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.921348077Z namespaces/openshift-monitoring/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.922328065Z namespaces/openshift-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.922460149Z namespaces/openshift-monitoring/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923044976Z namespaces/openshift-monitoring/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923220611Z namespaces/openshift-monitoring/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923243251Z namespaces/openshift-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923406036Z namespaces/openshift-monitoring/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923450297Z namespaces/openshift-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923491499Z namespaces/openshift-monitoring/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92354962Z namespaces/openshift-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923633973Z namespaces/openshift-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923729696Z namespaces/openshift-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923773257Z namespaces/openshift-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923833759Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.923964942Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924001093Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/main.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924087776Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924120937Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924220279Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924267511Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/alertmanager-main-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924369434Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/cluster-monitoring-operator-prometheus-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924536849Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kube-state-metrics-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924644932Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kubernetes-monitoring-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.924958291Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/node-exporter-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925099925Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-prometheus-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925229948Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-thanos-sidecar-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925322491Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-operator-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925441765Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/telemetry.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925533937Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/thanos-querier.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925613119Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92564852Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/alertmanager-main.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925741383Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/cluster-monitoring-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925857266Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kube-state-metrics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.925947719Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kubelet.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926048662Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/metrics-server.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926133385Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/node-exporter.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926284149Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/openshift-state-metrics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926374311Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926462854Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926562047Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/telemeter-client.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926648789Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-querier.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926742712Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-sidecar.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926906287Z namespaces/openshift-monitoring/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.926960508Z namespaces/openshift-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92702351Z namespaces/openshift-monitoring/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92703168Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.927091562Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.927332499Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.927630037Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.927779412Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928047199Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928261266Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928303817Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/openshift-cluster-monitoring.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92840375Z namespaces/openshift-monitoring/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92841309Z namespaces/openshift-monitoring/pods/alertmanager-main-0/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928498372Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager-main-0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928673877Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928682227Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928686618Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928729679Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928849962Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928918314Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928938135Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928956375Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.928963195Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929005097Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92911551Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929186262Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929239903Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929253664Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929259164Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929269014Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929368407Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929441699Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92945924Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92946712Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9294723Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929523802Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929630155Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929701617Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929726088Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929732908Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929745898Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.92980982Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929905453Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.929974225Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930004405Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930009516Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930015696Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930069467Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93016035Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930229652Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930267433Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930277673Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930282044Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930314714Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930405587Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930473589Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93051723Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930543131Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/cluster-monitoring-operator-64786d9cbb-b2trz.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930642924Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/cluster-monitoring-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930650124Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/cluster-monitoring-operator/cluster-monitoring-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930654294Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/cluster-monitoring-operator/cluster-monitoring-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.930696025Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/cluster-monitoring-operator/cluster-monitoring-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931395505Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931468607Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-64786d9cbb-b2trz/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931515859Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931581411Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-state-metrics-7c9dbb8b7d-z4jfq.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931688804Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-main/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931697194Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931702064Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931747635Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931865779Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931941911Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931980702Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-self/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.931992413Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932004753Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932190998Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932203459Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932341543Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932517577Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-state-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932562289Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-state-metrics/kube-state-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932567599Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-state-metrics/kube-state-metrics/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932581439Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-state-metrics/kube-state-metrics/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932586519Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-state-metrics/kube-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.932867458Z namespaces/openshift-monitoring/pods/kube-state-metrics-7c9dbb8b7d-z4jfq/kube-state-metrics/kube-state-metrics/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933008072Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933111325Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/metrics-server-689b57858d-fr7bt.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933116315Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/metrics-server/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933211608Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/metrics-server/metrics-server/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93328291Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/metrics-server/metrics-server/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93328773Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/metrics-server/metrics-server/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93331756Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/metrics-server/metrics-server/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933345071Z namespaces/openshift-monitoring/pods/metrics-server-689b57858d-fr7bt/metrics-server/metrics-server/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933350521Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933355752Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/monitoring-plugin-d5dcff7cf-wjsnk.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933361032Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/monitoring-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933366342Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/monitoring-plugin/monitoring-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933372802Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/monitoring-plugin/monitoring-plugin/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933377332Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/monitoring-plugin/monitoring-plugin/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933390583Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/monitoring-plugin/monitoring-plugin/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933458344Z namespaces/openshift-monitoring/pods/monitoring-plugin-d5dcff7cf-wjsnk/monitoring-plugin/monitoring-plugin/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933504476Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933550677Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/node-exporter-rkxgm.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9336428Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/init-textfile/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93365215Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/init-textfile/init-textfile/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93365854Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/init-textfile/init-textfile/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933705002Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933776294Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933867816Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933905267Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933911448Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933919908Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.933967599Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934056232Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934120103Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934169915Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/node-exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934177525Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/node-exporter/node-exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934183585Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/node-exporter/node-exporter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934231257Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93433884Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934417692Z namespaces/openshift-monitoring/pods/node-exporter-rkxgm/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934466084Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934509715Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/node-exporter-v89gr.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934717551Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/init-textfile/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934728701Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/init-textfile/init-textfile/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934733081Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/init-textfile/init-textfile/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934807103Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934941337Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.934999329Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935033Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93504035Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9350445Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935079511Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935167793Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935239046Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935287877Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/node-exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935301207Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/node-exporter/node-exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935310798Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/node-exporter/node-exporter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935610026Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935621686Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935657987Z namespaces/openshift-monitoring/pods/node-exporter-v89gr/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935711129Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935718309Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/node-exporter-wknkb.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93573137Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/init-textfile/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9357357Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/init-textfile/init-textfile/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93573976Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/init-textfile/init-textfile/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93575016Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935833583Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935903494Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935946456Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935956086Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935962176Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.935976887Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93607839Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936284556Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936301156Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/node-exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936305876Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/node-exporter/node-exporter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936309726Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/node-exporter/node-exporter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936320037Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936352147Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93643637Z namespaces/openshift-monitoring/pods/node-exporter-wknkb/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936483201Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936516782Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/openshift-state-metrics-54cf96dfcf-8xf64.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936663207Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-main/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936671927Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936678117Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936713198Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936812001Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936881663Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936921864Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-self/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936931254Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936940904Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.936985786Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937082299Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937192791Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937210692Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/openshift-state-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937276804Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/openshift-state-metrics/openshift-state-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937284454Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/openshift-state-metrics/openshift-state-metrics/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937330735Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/openshift-state-metrics/openshift-state-metrics/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937344636Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/openshift-state-metrics/openshift-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93747262Z namespaces/openshift-monitoring/pods/openshift-state-metrics-54cf96dfcf-8xf64/openshift-state-metrics/openshift-state-metrics/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93747951Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937571442Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus-k8s-0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937777088Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937807589Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93782369Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.937893532Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938014365Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938086157Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938121178Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938128538Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938134878Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93818536Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938274693Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938343255Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938367095Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938372105Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938376745Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938421827Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938509769Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938576701Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938609422Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938616152Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938620072Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938653893Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938745076Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938834689Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93888912Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938897Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938901331Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.938942962Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939024914Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939090406Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939129187Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939136827Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939140918Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939513808Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939764595Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939856898Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939902759Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93991498Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.93992152Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.939956931Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940066324Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940129016Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940176137Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940202948Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/prometheus-operator-567f7b8b7c-l9xd4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940304011Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940311251Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940315561Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940352942Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940457645Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940528227Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940567178Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/prometheus-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940574649Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/prometheus-operator/prometheus-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940578839Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/prometheus-operator/prometheus-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94061719Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/prometheus-operator/prometheus-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940913518Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/prometheus-operator/prometheus-operator/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.940934829Z namespaces/openshift-monitoring/pods/prometheus-operator-567f7b8b7c-l9xd4/prometheus-operator/prometheus-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94097922Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941032932Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/prometheus-operator-admission-webhook-d559d66d5-kswvz.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941131695Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/prometheus-operator-admission-webhook/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941144115Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941152475Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941169016Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941279459Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941346791Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-d559d66d5-kswvz/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941366501Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941435613Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/telemeter-client-6bc7f8c49d-shxfk.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941594818Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941604948Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941612348Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94167508Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941749082Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941840635Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941863236Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/reload/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941871516Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/reload/reload/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941875606Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/reload/reload/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.941933507Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/reload/reload/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9420218Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/reload/reload/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942091302Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/reload/reload/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942142313Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/telemeter-client/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942153504Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/telemeter-client/telemeter-client/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942158154Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/telemeter-client/telemeter-client/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942174565Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/telemeter-client/telemeter-client/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942330029Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/telemeter-client/telemeter-client/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94238268Z namespaces/openshift-monitoring/pods/telemeter-client-6bc7f8c49d-shxfk/telemeter-client/telemeter-client/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942419541Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942489614Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/thanos-querier-5bb78d8ff5-z27bl.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942624437Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942632878Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942637498Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942667989Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942759531Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.942905166Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943012198Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-rules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943022209Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-rules/kube-rbac-proxy-rules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943026379Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94305889Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943150733Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943218715Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943240975Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-web/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943250715Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943259506Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943313297Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943558114Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943621476Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943656757Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943663527Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943667727Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943711709Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943815752Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943888644Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943933155Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/prom-label-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943943445Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943948525Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.943966726Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944063859Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94412389Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944171702Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/thanos-query/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944181142Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/thanos-query/thanos-query/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944188822Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/thanos-query/thanos-query/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944210233Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/thanos-query/thanos-query/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944319666Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/thanos-query/thanos-query/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944384898Z namespaces/openshift-monitoring/pods/thanos-querier-5bb78d8ff5-z27bl/thanos-query/thanos-query/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944419359Z namespaces/openshift-monitoring/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94446239Z namespaces/openshift-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944495651Z namespaces/openshift-monitoring/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944543622Z namespaces/openshift-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944667516Z namespaces/openshift-multus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944712137Z namespaces/openshift-multus/openshift-multus.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944786729Z namespaces/openshift-multus/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944860431Z namespaces/openshift-multus/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944927204Z namespaces/openshift-multus/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.944964074Z namespaces/openshift-multus/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945240022Z namespaces/openshift-multus/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945315215Z namespaces/openshift-multus/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945401347Z namespaces/openshift-multus/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945439438Z namespaces/openshift-multus/apps/daemonsets/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945487429Z namespaces/openshift-multus/apps/daemonsets/multus-additional-cni-plugins.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945636394Z namespaces/openshift-multus/apps/daemonsets/multus.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945754707Z namespaces/openshift-multus/apps/daemonsets/network-metrics-daemon.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945828959Z namespaces/openshift-multus/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945877321Z namespaces/openshift-multus/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945923902Z namespaces/openshift-multus/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.945964003Z namespaces/openshift-multus/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.946043136Z namespaces/openshift-multus/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.946087307Z namespaces/openshift-multus/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.946131298Z namespaces/openshift-multus/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94621518Z namespaces/openshift-multus/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.946272582Z namespaces/openshift-multus/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.946314773Z namespaces/openshift-multus/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.946459507Z namespaces/openshift-multus/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94654316Z namespaces/openshift-multus/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.947101886Z namespaces/openshift-multus/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.947220819Z namespaces/openshift-multus/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.947855047Z namespaces/openshift-multus/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94793531Z namespaces/openshift-multus/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948171717Z namespaces/openshift-multus/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948217048Z namespaces/openshift-multus/core/configmaps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948265999Z namespaces/openshift-multus/core/configmaps/cni-copy-resources.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948369842Z namespaces/openshift-multus/core/configmaps/default-cni-sysctl-allowlist.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948450284Z namespaces/openshift-multus/core/configmaps/multus-daemon-config.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948534317Z namespaces/openshift-multus/core/configmaps/whereabouts-flatfile-config.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948628699Z namespaces/openshift-multus/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948675391Z namespaces/openshift-multus/core/serviceaccounts/metrics-daemon-sa.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948815675Z namespaces/openshift-multus/core/serviceaccounts/multus-ac.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.948909498Z namespaces/openshift-multus/core/serviceaccounts/multus-ancillary-tools.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94898974Z namespaces/openshift-multus/core/serviceaccounts/multus.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949043171Z namespaces/openshift-multus/core/services/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949088193Z namespaces/openshift-multus/core/services/network-metrics-service.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949145574Z namespaces/openshift-multus/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949198596Z namespaces/openshift-multus/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949247597Z namespaces/openshift-multus/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949291389Z namespaces/openshift-multus/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94933902Z namespaces/openshift-multus/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949385561Z namespaces/openshift-multus/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949462243Z namespaces/openshift-multus/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949546856Z namespaces/openshift-multus/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949601187Z namespaces/openshift-multus/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949643139Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.94969831Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949743002Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/monitor-network.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949806353Z namespaces/openshift-multus/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949848934Z namespaces/openshift-multus/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949900486Z namespaces/openshift-multus/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949918067Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.949971788Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.950117632Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.950295927Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.950429371Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.950644587Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.950856813Z namespaces/openshift-multus/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.950870384Z namespaces/openshift-multus/pods/multus-9jwq4/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.950918785Z namespaces/openshift-multus/pods/multus-9jwq4/multus-9jwq4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.951026178Z namespaces/openshift-multus/pods/multus-9jwq4/kube-multus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.951034988Z namespaces/openshift-multus/pods/multus-9jwq4/kube-multus/kube-multus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.951048809Z namespaces/openshift-multus/pods/multus-9jwq4/kube-multus/kube-multus/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95108125Z namespaces/openshift-multus/pods/multus-9jwq4/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.951989616Z namespaces/openshift-multus/pods/multus-9jwq4/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952059828Z namespaces/openshift-multus/pods/multus-9jwq4/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952082109Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95213662Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/multus-additional-cni-plugins-79hbm.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952289655Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/bond-cni-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952298545Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952302645Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952331296Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952431788Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95249577Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952535442Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952542542Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/cni-plugins/cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952546772Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952588903Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952686216Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952773898Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95284143Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/egress-router-binary-copy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952859651Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952870921Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.952921473Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953020176Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953090818Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953116208Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953121838Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953132319Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95317708Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953249072Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953311524Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953350685Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/routeoverride-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953358495Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953362975Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953421337Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953551181Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953605892Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953651454Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni-bincopy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953664824Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953674804Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953689895Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953809558Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95386763Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953906341Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953921241Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953928102Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.953952382Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954041575Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954110567Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-79hbm/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95457639Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954635712Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/multus-additional-cni-plugins-xqmj9.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954764675Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/bond-cni-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954780406Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954786526Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954822767Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95490898Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.954981252Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955002272Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955010253Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/cni-plugins/cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955022003Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955057064Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955153207Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955215199Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955233159Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/egress-router-binary-copy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955239449Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955249689Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955309761Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955400924Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955490736Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955530248Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955539418Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955545478Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955588959Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955684622Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955757414Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955778215Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/routeoverride-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955784445Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.955807336Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95596108Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956046072Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956114984Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956134365Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni-bincopy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956139805Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956148925Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956194556Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956290249Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956357231Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956385552Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956393672Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956401773Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956454464Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956537996Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956610808Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-xqmj9/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95667416Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956720942Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/multus-additional-cni-plugins-z4q2s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956873046Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/bond-cni-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956894227Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956899327Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.956913757Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95701357Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957070532Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957087302Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957093472Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/cni-plugins/cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957099362Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957142044Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957234826Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957299018Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957339459Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/egress-router-binary-copy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9573519Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95735674Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957383291Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957476413Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957540675Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957586147Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957598237Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957603687Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957619717Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957690379Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957761912Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957786342Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/routeoverride-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957819063Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957825063Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957862304Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.957954467Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958017089Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958034199Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni-bincopy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95804467Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.95805044Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958092571Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958188954Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958254506Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958286607Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958295597Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958299717Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958335388Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958431361Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958495613Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-z4q2s/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958512203Z namespaces/openshift-multus/pods/multus-pgnpv/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958567704Z namespaces/openshift-multus/pods/multus-pgnpv/multus-pgnpv.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958662747Z namespaces/openshift-multus/pods/multus-pgnpv/kube-multus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958673648Z namespaces/openshift-multus/pods/multus-pgnpv/kube-multus/kube-multus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958682988Z namespaces/openshift-multus/pods/multus-pgnpv/kube-multus/kube-multus/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.958694888Z namespaces/openshift-multus/pods/multus-pgnpv/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959250534Z namespaces/openshift-multus/pods/multus-pgnpv/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959336027Z namespaces/openshift-multus/pods/multus-pgnpv/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959349387Z namespaces/openshift-multus/pods/multus-sp6j6/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959406939Z namespaces/openshift-multus/pods/multus-sp6j6/multus-sp6j6.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959493601Z namespaces/openshift-multus/pods/multus-sp6j6/kube-multus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959502011Z namespaces/openshift-multus/pods/multus-sp6j6/kube-multus/kube-multus/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959508692Z namespaces/openshift-multus/pods/multus-sp6j6/kube-multus/kube-multus/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.959588524Z namespaces/openshift-multus/pods/multus-sp6j6/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960467889Z namespaces/openshift-multus/pods/multus-sp6j6/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960538081Z namespaces/openshift-multus/pods/multus-sp6j6/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960551212Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960606363Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/network-metrics-daemon-bgqxw.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960699146Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960708896Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960715086Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960750357Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960870521Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960939612Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960953273Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/network-metrics-daemon/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960975124Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.960994524Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961005194Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961245861Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961317323Z namespaces/openshift-multus/pods/network-metrics-daemon-bgqxw/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961341004Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961394865Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/network-metrics-daemon-cnnqh.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961477338Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961491288Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961496179Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961511139Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961609942Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961681374Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961727885Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/network-metrics-daemon/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961737785Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961743426Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961777407Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.961966332Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962039334Z namespaces/openshift-multus/pods/network-metrics-daemon-cnnqh/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962065055Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962112546Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/network-metrics-daemon-pwbrd.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962190208Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962202959Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962210299Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962228449Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962309482Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962372904Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962424755Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/network-metrics-daemon/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962434326Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962441016Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962451016Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96260167Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962669822Z namespaces/openshift-multus/pods/network-metrics-daemon-pwbrd/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962692213Z namespaces/openshift-multus/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962744174Z namespaces/openshift-multus/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962809936Z namespaces/openshift-multus/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.962834567Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9629286Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/multus-whereabouts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963041783Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963075714Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963153156Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963239438Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/whereabouts-cni.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96328689Z namespaces/openshift-multus/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963344201Z namespaces/openshift-multus/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963365482Z namespaces/openshift-must-gather-h4rcg/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963372572Z namespaces/openshift-must-gather-h4rcg/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963379322Z namespaces/openshift-must-gather-h4rcg/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963437454Z namespaces/openshift-must-gather-h4rcg/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963600049Z namespaces/openshift-must-gather-h4rcg/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963775334Z namespaces/openshift-must-gather-h4rcg/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.963932428Z namespaces/openshift-must-gather-h4rcg/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964150725Z namespaces/openshift-must-gather-h4rcg/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96431954Z namespaces/openshift-network-console/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964366151Z namespaces/openshift-network-console/openshift-network-console.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964416582Z namespaces/openshift-network-console/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964459413Z namespaces/openshift-network-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964506155Z namespaces/openshift-network-console/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964547736Z namespaces/openshift-network-console/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964629589Z namespaces/openshift-network-console/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964743602Z namespaces/openshift-network-console/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964861715Z namespaces/openshift-network-console/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964904006Z namespaces/openshift-network-console/apps/deployments/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.964946538Z namespaces/openshift-network-console/apps/deployments/networking-console-plugin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96502361Z namespaces/openshift-network-console/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965070891Z namespaces/openshift-network-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965103442Z namespaces/openshift-network-console/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965142223Z namespaces/openshift-network-console/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965218445Z namespaces/openshift-network-console/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965272257Z namespaces/openshift-network-console/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965303838Z namespaces/openshift-network-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96538614Z namespaces/openshift-network-console/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965419101Z namespaces/openshift-network-console/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965463292Z namespaces/openshift-network-console/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965572475Z namespaces/openshift-network-console/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965658988Z namespaces/openshift-network-console/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965781521Z namespaces/openshift-network-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.965872484Z namespaces/openshift-network-console/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966049549Z namespaces/openshift-network-console/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966189763Z namespaces/openshift-network-console/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966317677Z namespaces/openshift-network-console/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966369548Z namespaces/openshift-network-console/core/configmaps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966400209Z namespaces/openshift-network-console/core/configmaps/networking-console-plugin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966564294Z namespaces/openshift-network-console/core/services/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966603745Z namespaces/openshift-network-console/core/services/networking-console-plugin.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966655496Z namespaces/openshift-network-console/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966685647Z namespaces/openshift-network-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966745609Z namespaces/openshift-network-console/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96677582Z namespaces/openshift-network-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966859452Z namespaces/openshift-network-console/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966904384Z namespaces/openshift-network-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.966986976Z namespaces/openshift-network-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967072499Z namespaces/openshift-network-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967103409Z namespaces/openshift-network-console/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96713792Z namespaces/openshift-network-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967188842Z namespaces/openshift-network-console/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967217023Z namespaces/openshift-network-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967283265Z namespaces/openshift-network-console/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967290915Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967317326Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96746841Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967642945Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.967781319Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968003885Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96818692Z namespaces/openshift-network-console/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968201051Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968211271Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/networking-console-plugin-6bffdf9778-6wrqd.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968300254Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/networking-console-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968311844Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/networking-console-plugin/networking-console-plugin/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968315874Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/networking-console-plugin/networking-console-plugin/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968335695Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/networking-console-plugin/networking-console-plugin/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968459148Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/networking-console-plugin/networking-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96852762Z namespaces/openshift-network-console/pods/networking-console-plugin-6bffdf9778-6wrqd/networking-console-plugin/networking-console-plugin/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968558861Z namespaces/openshift-network-console/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968642964Z namespaces/openshift-network-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968694515Z namespaces/openshift-network-console/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968749167Z namespaces/openshift-network-console/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968769597Z namespaces/openshift-network-diagnostics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96886188Z namespaces/openshift-network-diagnostics/openshift-network-diagnostics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968921801Z namespaces/openshift-network-diagnostics/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.968967283Z namespaces/openshift-network-diagnostics/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969000484Z namespaces/openshift-network-diagnostics/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969034325Z namespaces/openshift-network-diagnostics/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969164168Z namespaces/openshift-network-diagnostics/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969289572Z namespaces/openshift-network-diagnostics/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969393075Z namespaces/openshift-network-diagnostics/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969435646Z namespaces/openshift-network-diagnostics/apps/daemonsets/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969476087Z namespaces/openshift-network-diagnostics/apps/daemonsets/network-check-target.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969537739Z namespaces/openshift-network-diagnostics/apps/deployments/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96958134Z namespaces/openshift-network-diagnostics/apps/deployments/network-check-source.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969653062Z namespaces/openshift-network-diagnostics/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969695254Z namespaces/openshift-network-diagnostics/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969737915Z namespaces/openshift-network-diagnostics/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969778506Z namespaces/openshift-network-diagnostics/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969877319Z namespaces/openshift-network-diagnostics/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.96991282Z namespaces/openshift-network-diagnostics/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.969946781Z namespaces/openshift-network-diagnostics/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.970036764Z namespaces/openshift-network-diagnostics/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.970078805Z namespaces/openshift-network-diagnostics/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.970133826Z namespaces/openshift-network-diagnostics/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.970248319Z namespaces/openshift-network-diagnostics/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.970340882Z namespaces/openshift-network-diagnostics/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.970526387Z namespaces/openshift-network-diagnostics/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97060435Z namespaces/openshift-network-diagnostics/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.970940559Z namespaces/openshift-network-diagnostics/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971031792Z namespaces/openshift-network-diagnostics/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971135545Z namespaces/openshift-network-diagnostics/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971180206Z namespaces/openshift-network-diagnostics/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971221907Z namespaces/openshift-network-diagnostics/core/serviceaccounts/network-diagnostics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971288249Z namespaces/openshift-network-diagnostics/core/services/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971340351Z namespaces/openshift-network-diagnostics/core/services/network-check-source.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971422663Z namespaces/openshift-network-diagnostics/core/services/network-check-target.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971472155Z namespaces/openshift-network-diagnostics/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971512376Z namespaces/openshift-network-diagnostics/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971559987Z namespaces/openshift-network-diagnostics/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971601138Z namespaces/openshift-network-diagnostics/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97164347Z namespaces/openshift-network-diagnostics/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971683451Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971764053Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971870486Z namespaces/openshift-network-diagnostics/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971921247Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.971943198Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97201268Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972034611Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/network-check-source.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972081402Z namespaces/openshift-network-diagnostics/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972109433Z namespaces/openshift-network-diagnostics/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972171115Z namespaces/openshift-network-diagnostics/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972184475Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972199846Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972383211Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972581586Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.972818413Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973034379Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973216775Z namespaces/openshift-network-diagnostics/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973230475Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973246296Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/network-check-source-989f8b8dc-x5thm.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973339618Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/check-endpoints/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973352248Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/check-endpoints/check-endpoints/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973359369Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/check-endpoints/check-endpoints/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973371779Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/check-endpoints/check-endpoints/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973488003Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/check-endpoints/check-endpoints/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973621626Z namespaces/openshift-network-diagnostics/pods/network-check-source-989f8b8dc-x5thm/check-endpoints/check-endpoints/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973643977Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973692078Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/network-check-target-77jbc.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973786791Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/network-check-target-container/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973828622Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973839082Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973851333Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.973993427Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974068679Z namespaces/openshift-network-diagnostics/pods/network-check-target-77jbc/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97409284Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974143181Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/network-check-target-kdc88.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974220913Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/network-check-target-container/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974233514Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974239954Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974250324Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974334697Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974400378Z namespaces/openshift-network-diagnostics/pods/network-check-target-kdc88/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974422799Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974471751Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/network-check-target-lrj5f.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974544183Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/network-check-target-container/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974553013Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974560353Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974569363Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974682787Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974732438Z namespaces/openshift-network-diagnostics/pods/network-check-target-lrj5f/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974778069Z namespaces/openshift-network-diagnostics/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97482105Z namespaces/openshift-network-diagnostics/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974877732Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974894313Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.974904713Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975004446Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975041957Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975086368Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/network-diagnostics.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97515325Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975210992Z namespaces/openshift-network-diagnostics/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975256733Z namespaces/openshift-network-diagnostics/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975304184Z namespaces/openshift-network-node-identity/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975344376Z namespaces/openshift-network-node-identity/openshift-network-node-identity.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975410607Z namespaces/openshift-network-node-identity/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975450319Z namespaces/openshift-network-node-identity/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97550369Z namespaces/openshift-network-node-identity/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975518701Z namespaces/openshift-network-node-identity/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975613673Z namespaces/openshift-network-node-identity/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975694016Z namespaces/openshift-network-node-identity/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975768198Z namespaces/openshift-network-node-identity/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97583913Z namespaces/openshift-network-node-identity/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97585623Z namespaces/openshift-network-node-identity/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975916122Z namespaces/openshift-network-node-identity/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.975954933Z namespaces/openshift-network-node-identity/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976041876Z namespaces/openshift-network-node-identity/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976081937Z namespaces/openshift-network-node-identity/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976131508Z namespaces/openshift-network-node-identity/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97620321Z namespaces/openshift-network-node-identity/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976257982Z namespaces/openshift-network-node-identity/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976266402Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976284573Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ovnkube-identity.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976353334Z namespaces/openshift-network-node-identity/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976373855Z namespaces/openshift-network-node-identity/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976497979Z namespaces/openshift-network-node-identity/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976591441Z namespaces/openshift-network-node-identity/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976702704Z namespaces/openshift-network-node-identity/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.976819588Z namespaces/openshift-network-node-identity/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97691143Z namespaces/openshift-network-node-identity/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977095026Z namespaces/openshift-network-node-identity/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977201089Z namespaces/openshift-network-node-identity/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97725065Z namespaces/openshift-network-node-identity/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977286661Z namespaces/openshift-network-node-identity/core/serviceaccounts/network-node-identity.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977342333Z namespaces/openshift-network-node-identity/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977385774Z namespaces/openshift-network-node-identity/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977431865Z namespaces/openshift-network-node-identity/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977473537Z namespaces/openshift-network-node-identity/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977519068Z namespaces/openshift-network-node-identity/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977560419Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977637762Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977718754Z namespaces/openshift-network-node-identity/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977762465Z namespaces/openshift-network-node-identity/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977805186Z namespaces/openshift-network-node-identity/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977867328Z namespaces/openshift-network-node-identity/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977909589Z namespaces/openshift-network-node-identity/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977956891Z namespaces/openshift-network-node-identity/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977962411Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.977998842Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.978142536Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.978321041Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.978457605Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.978657631Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.978931199Z namespaces/openshift-network-node-identity/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97896576Z namespaces/openshift-network-node-identity/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979022721Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979029911Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979056102Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/network-node-identity-leases.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979114424Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979157375Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/network-node-identity-leases.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979205156Z namespaces/openshift-network-node-identity/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979253128Z namespaces/openshift-network-node-identity/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979299449Z namespaces/openshift-network-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.97935238Z namespaces/openshift-network-operator/openshift-network-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979414702Z namespaces/openshift-network-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979435093Z namespaces/openshift-network-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979491985Z namespaces/openshift-network-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979535266Z namespaces/openshift-network-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979639759Z namespaces/openshift-network-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979710661Z namespaces/openshift-network-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979815194Z namespaces/openshift-network-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979846495Z namespaces/openshift-network-operator/apps/daemonsets/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979906607Z namespaces/openshift-network-operator/apps/daemonsets/iptables-alerter.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.979975038Z namespaces/openshift-network-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98002864Z namespaces/openshift-network-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980073611Z namespaces/openshift-network-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980149014Z namespaces/openshift-network-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980259037Z namespaces/openshift-network-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980308128Z namespaces/openshift-network-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980349249Z namespaces/openshift-network-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980427811Z namespaces/openshift-network-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980471433Z namespaces/openshift-network-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980478553Z namespaces/openshift-network-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980507124Z namespaces/openshift-network-operator/coordination.k8s.io/leases/network-operator-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980552985Z namespaces/openshift-network-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980601576Z namespaces/openshift-network-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98072515Z namespaces/openshift-network-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980804502Z namespaces/openshift-network-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.980991218Z namespaces/openshift-network-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98106909Z namespaces/openshift-network-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981242785Z namespaces/openshift-network-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981332377Z namespaces/openshift-network-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981466891Z namespaces/openshift-network-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981535823Z namespaces/openshift-network-operator/core/configmaps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981581364Z namespaces/openshift-network-operator/core/configmaps/applied-cluster.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981676567Z namespaces/openshift-network-operator/core/configmaps/iptables-alerter-script.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981748459Z namespaces/openshift-network-operator/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981802561Z namespaces/openshift-network-operator/core/serviceaccounts/iptables-alerter.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981874533Z namespaces/openshift-network-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981921304Z namespaces/openshift-network-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.981993556Z namespaces/openshift-network-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982037738Z namespaces/openshift-network-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982091509Z namespaces/openshift-network-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982140711Z namespaces/openshift-network-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982228003Z namespaces/openshift-network-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982297285Z namespaces/openshift-network-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982355987Z namespaces/openshift-network-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982369497Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982438379Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98248001Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/openshift-network-operator-ipsec-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982525911Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982567283Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/network-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982623674Z namespaces/openshift-network-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982651715Z namespaces/openshift-network-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982708837Z namespaces/openshift-network-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982716427Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982743418Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.982915983Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983113548Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983263973Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983495339Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983687045Z namespaces/openshift-network-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983696635Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983733426Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/iptables-alerter-jt7n8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983834449Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/iptables-alerter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98384532Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98384933Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983886991Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.983978823Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984040485Z namespaces/openshift-network-operator/pods/iptables-alerter-jt7n8/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984076736Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984115407Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/iptables-alerter-kgjn6.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984188059Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/iptables-alerter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984196559Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984201Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98422533Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984330683Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984394465Z namespaces/openshift-network-operator/pods/iptables-alerter-kgjn6/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984411906Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984459237Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/iptables-alerter-x2w2v.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984534199Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/iptables-alerter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984544749Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98455031Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98456614Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984655413Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984718925Z namespaces/openshift-network-operator/pods/iptables-alerter-x2w2v/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984735855Z namespaces/openshift-network-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984815767Z namespaces/openshift-network-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984874279Z namespaces/openshift-network-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984893479Z namespaces/openshift-network-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984958761Z namespaces/openshift-node/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984970102Z namespaces/openshift-node/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.984975052Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.985019893Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.985186808Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.985372133Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.985523178Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.985909259Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986093764Z namespaces/openshift-operator-lifecycle-manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986126865Z namespaces/openshift-operator-lifecycle-manager/openshift-operator-lifecycle-manager.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986195897Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986227258Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986279859Z namespaces/openshift-operator-lifecycle-manager/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98631129Z namespaces/openshift-operator-lifecycle-manager/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986415843Z namespaces/openshift-operator-lifecycle-manager/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986494476Z namespaces/openshift-operator-lifecycle-manager/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986582448Z namespaces/openshift-operator-lifecycle-manager/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986617059Z namespaces/openshift-operator-lifecycle-manager/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986672041Z namespaces/openshift-operator-lifecycle-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986721712Z namespaces/openshift-operator-lifecycle-manager/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.986771743Z namespaces/openshift-operator-lifecycle-manager/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9869964Z namespaces/openshift-operator-lifecycle-manager/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987050961Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987097703Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987191245Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987244847Z namespaces/openshift-operator-lifecycle-manager/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987309889Z namespaces/openshift-operator-lifecycle-manager/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987436272Z namespaces/openshift-operator-lifecycle-manager/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987518945Z namespaces/openshift-operator-lifecycle-manager/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987613377Z namespaces/openshift-operator-lifecycle-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98768663Z namespaces/openshift-operator-lifecycle-manager/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987768382Z namespaces/openshift-operator-lifecycle-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.987905266Z namespaces/openshift-operator-lifecycle-manager/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988133592Z namespaces/openshift-operator-lifecycle-manager/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988205314Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988260406Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988313648Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988352729Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98841052Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988432671Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988540984Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988626097Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988665748Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988710159Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98876287Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988827832Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/olm-alert-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988908495Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.988956056Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.989037888Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.989045418Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9890888Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.989251984Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.98942758Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.989560463Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.989761399Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.989980055Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990019866Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/olm-operators.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990077938Z namespaces/openshift-operator-lifecycle-manager/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990109249Z namespaces/openshift-operator-lifecycle-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990169051Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990204752Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990250953Z namespaces/openshift-operators/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990260173Z namespaces/openshift-operators/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990264633Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990317155Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99047806Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.990664555Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99084266Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991078257Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991380956Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991423907Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/global-operators.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991466498Z namespaces/openshift-ovn-kubernetes/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991507529Z namespaces/openshift-ovn-kubernetes/openshift-ovn-kubernetes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991579911Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991622603Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991664594Z namespaces/openshift-ovn-kubernetes/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991705705Z namespaces/openshift-ovn-kubernetes/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99189837Z namespaces/openshift-ovn-kubernetes/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.991993343Z namespaces/openshift-ovn-kubernetes/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992078925Z namespaces/openshift-ovn-kubernetes/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992122577Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992163628Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ovnkube-node.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992307672Z namespaces/openshift-ovn-kubernetes/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992349693Z namespaces/openshift-ovn-kubernetes/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992391365Z namespaces/openshift-ovn-kubernetes/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992453636Z namespaces/openshift-ovn-kubernetes/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992546809Z namespaces/openshift-ovn-kubernetes/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9925914Z namespaces/openshift-ovn-kubernetes/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992634441Z namespaces/openshift-ovn-kubernetes/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992740864Z namespaces/openshift-ovn-kubernetes/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992785566Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992813407Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.992876539Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ovn-kubernetes-master.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99291756Z namespaces/openshift-ovn-kubernetes/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99294835Z namespaces/openshift-ovn-kubernetes/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.993161917Z namespaces/openshift-ovn-kubernetes/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.993240149Z namespaces/openshift-ovn-kubernetes/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.993555828Z namespaces/openshift-ovn-kubernetes/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.993686122Z namespaces/openshift-ovn-kubernetes/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994251378Z namespaces/openshift-ovn-kubernetes/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994356371Z namespaces/openshift-ovn-kubernetes/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994520806Z namespaces/openshift-ovn-kubernetes/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994567347Z namespaces/openshift-ovn-kubernetes/core/configmaps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994597588Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-config.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994691291Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-script-lib.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994800604Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.994952598Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-control-plane.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.9950329Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-node.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995075282Z namespaces/openshift-ovn-kubernetes/core/services/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995121143Z namespaces/openshift-ovn-kubernetes/core/services/ovn-kubernetes-node.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995176254Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995234986Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995323529Z namespaces/openshift-ovn-kubernetes/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99536465Z namespaces/openshift-ovn-kubernetes/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995425652Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995433032Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995472773Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/default.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995535605Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995570046Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995654488Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99574127Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995806683Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995886765Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995949776Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.995972437Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/networking-rules.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99607764Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996112991Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/monitor-ovn-node.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996291476Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996305527Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996315807Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ovn.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99640206Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/signer.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996451381Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996481802Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996541484Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996552564Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996569654Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996736689Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.996939655Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997060428Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997285215Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99746196Z namespaces/openshift-ovn-kubernetes/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99747324Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997541812Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovnkube-node-22jbf.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997700087Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-node/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997708807Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997712897Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997755428Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997898323Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997962414Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997983095Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.997991595Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998000375Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998041467Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99816792Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998230402Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998281054Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/nbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998290224Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/nbdb/nbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998295354Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/nbdb/nbdb/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998303684Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/nbdb/nbdb/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998411507Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998479789Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99849278Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/northd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99849917Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/northd/northd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.99851038Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/northd/northd/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998556401Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/northd/northd/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998655194Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998726096Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/northd/northd/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998745007Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-acl-logging/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998765947Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998771768Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998779098Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998901971Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-acl-logging/ovn-acl-logging/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.998969163Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999012295Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999023565Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-controller/ovn-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999028075Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999037165Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999359444Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999425676Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999457827Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovnkube-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999462557Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999515349Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:33.999542459Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001220638Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00128965Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00130678Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/sbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00131464Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/sbdb/sbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00131975Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/sbdb/sbdb/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001375242Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/sbdb/sbdb/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001468955Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001537897Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-22jbf/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001552567Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00164594Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovnkube-node-ndw9q.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001808944Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-node/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001823025Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001828185Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001867036Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.001975119Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002037941Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002055872Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002064172Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002076392Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002114253Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002232297Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002285028Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00233067Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/nbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00234926Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/nbdb/nbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00235525Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/nbdb/nbdb/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002366491Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/nbdb/nbdb/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002456183Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002532665Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002551576Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/northd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002559406Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/northd/northd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002563746Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/northd/northd/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002600597Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/northd/northd/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00271011Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002777742Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/northd/northd/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002827874Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-acl-logging/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002851554Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002858755Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002869475Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.002967148Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-acl-logging/ovn-acl-logging/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00304035Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00305372Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003063971Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-controller/ovn-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003069111Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003114072Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00338517Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003436791Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003472802Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovnkube-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003481152Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003488693Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.003521464Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005087529Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00515164Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005187092Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/sbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005196082Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/sbdb/sbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005201002Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/sbdb/sbdb/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005219552Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/sbdb/sbdb/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005339426Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005389877Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-ndw9q/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005421088Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005525561Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovnkube-node-sntx9.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005678346Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-node/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005689616Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005694776Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005716837Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00584686Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005912612Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005934483Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005943663Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005954803Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.005992235Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006107228Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00617293Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00619338Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/nbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00620129Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/nbdb/nbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006205861Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/nbdb/nbdb/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006255582Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/nbdb/nbdb/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006349215Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006415987Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006429437Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/northd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006447677Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/northd/northd/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006461938Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/northd/northd/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006472418Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/northd/northd/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006599682Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006665114Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/northd/northd/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006678964Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-acl-logging/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006684194Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006697635Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006737316Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006846099Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00689419Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006902351Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-controller/ovn-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006907111Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.006943802Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.007342893Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.007403785Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.007421965Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovnkube-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.007428336Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.007434096Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.007475527Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009233848Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009291859Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009339431Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/sbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009349021Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/sbdb/sbdb/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009354391Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/sbdb/sbdb/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009363231Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/sbdb/sbdb/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009461844Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009541856Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-sntx9/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009555607Z namespaces/openshift-ovn-kubernetes/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009604138Z namespaces/openshift-ovn-kubernetes/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00966193Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00967177Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.00968545Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009816214Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-nodes-identity-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009932768Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.009968969Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010002299Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010097722Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010176844Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010234396Z namespaces/openshift-ovn-kubernetes/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010254667Z namespaces/openshift-ovn-kubernetes/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010316408Z namespaces/openshift-route-controller-manager/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010327469Z namespaces/openshift-route-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010334929Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010343629Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/openshift-route-controllers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010404971Z namespaces/openshift-route-controller-manager/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010414881Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010458303Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010651648Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.010892625Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011037049Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011279356Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011491772Z namespaces/openshift-service-ca-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011534923Z namespaces/openshift-service-ca-operator/openshift-service-ca-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011600465Z namespaces/openshift-service-ca-operator/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011630996Z namespaces/openshift-service-ca-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011686358Z namespaces/openshift-service-ca-operator/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011717029Z namespaces/openshift-service-ca-operator/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.011811531Z namespaces/openshift-service-ca-operator/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012041848Z namespaces/openshift-service-ca-operator/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.0121288Z namespaces/openshift-service-ca-operator/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012171502Z namespaces/openshift-service-ca-operator/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012218783Z namespaces/openshift-service-ca-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012253124Z namespaces/openshift-service-ca-operator/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012303156Z namespaces/openshift-service-ca-operator/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012384168Z namespaces/openshift-service-ca-operator/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012419709Z namespaces/openshift-service-ca-operator/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01246702Z namespaces/openshift-service-ca-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012553023Z namespaces/openshift-service-ca-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012595624Z namespaces/openshift-service-ca-operator/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012607764Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012640105Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/service-ca-operator-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012693447Z namespaces/openshift-service-ca-operator/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012724928Z namespaces/openshift-service-ca-operator/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012858872Z namespaces/openshift-service-ca-operator/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.012949434Z namespaces/openshift-service-ca-operator/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013132149Z namespaces/openshift-service-ca-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013205551Z namespaces/openshift-service-ca-operator/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013346665Z namespaces/openshift-service-ca-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013474769Z namespaces/openshift-service-ca-operator/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013642114Z namespaces/openshift-service-ca-operator/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013699225Z namespaces/openshift-service-ca-operator/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013730546Z namespaces/openshift-service-ca-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013803899Z namespaces/openshift-service-ca-operator/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01385442Z namespaces/openshift-service-ca-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013902601Z namespaces/openshift-service-ca-operator/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.013943863Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014027285Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014103587Z namespaces/openshift-service-ca-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014140898Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014180709Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014237061Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014268172Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/service-ca-operator.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014320233Z namespaces/openshift-service-ca-operator/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014353034Z namespaces/openshift-service-ca-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014393685Z namespaces/openshift-service-ca-operator/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014401816Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014458747Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.014598061Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01489859Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015034474Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015281061Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015456596Z namespaces/openshift-service-ca-operator/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015467056Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5d46d9ffd4-j4cq4/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015503847Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5d46d9ffd4-j4cq4/service-ca-operator-5d46d9ffd4-j4cq4.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01559963Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5d46d9ffd4-j4cq4/service-ca-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015615611Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5d46d9ffd4-j4cq4/service-ca-operator/service-ca-operator/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015622241Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5d46d9ffd4-j4cq4/service-ca-operator/service-ca-operator/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015638561Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5d46d9ffd4-j4cq4/service-ca-operator/service-ca-operator/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015775305Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5d46d9ffd4-j4cq4/service-ca-operator/service-ca-operator/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01594435Z namespaces/openshift-service-ca-operator/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.015981391Z namespaces/openshift-service-ca-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016015492Z namespaces/openshift-service-ca-operator/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016070893Z namespaces/openshift-service-ca-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016114175Z namespaces/openshift-service-ca/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016160646Z namespaces/openshift-service-ca/openshift-service-ca.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016202797Z namespaces/openshift-service-ca/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016255159Z namespaces/openshift-service-ca/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01629833Z namespaces/openshift-service-ca/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016344181Z namespaces/openshift-service-ca/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016423464Z namespaces/openshift-service-ca/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016524357Z namespaces/openshift-service-ca/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016618719Z namespaces/openshift-service-ca/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016666001Z namespaces/openshift-service-ca/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016707142Z namespaces/openshift-service-ca/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016753153Z namespaces/openshift-service-ca/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016810025Z namespaces/openshift-service-ca/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016894077Z namespaces/openshift-service-ca/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.016932148Z namespaces/openshift-service-ca/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01698186Z namespaces/openshift-service-ca/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017061772Z namespaces/openshift-service-ca/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017111853Z namespaces/openshift-service-ca/coordination.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017120714Z namespaces/openshift-service-ca/coordination.k8s.io/leases/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017147735Z namespaces/openshift-service-ca/coordination.k8s.io/leases/service-ca-controller-lock.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017202566Z namespaces/openshift-service-ca/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017243457Z namespaces/openshift-service-ca/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01734974Z namespaces/openshift-service-ca/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017426082Z namespaces/openshift-service-ca/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017537746Z namespaces/openshift-service-ca/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017614958Z namespaces/openshift-service-ca/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017734791Z namespaces/openshift-service-ca/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.017858305Z namespaces/openshift-service-ca/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01802684Z namespaces/openshift-service-ca/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018078491Z namespaces/openshift-service-ca/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018122692Z namespaces/openshift-service-ca/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018169594Z namespaces/openshift-service-ca/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018212595Z namespaces/openshift-service-ca/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018255996Z namespaces/openshift-service-ca/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018298017Z namespaces/openshift-service-ca/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.01837516Z namespaces/openshift-service-ca/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018452912Z namespaces/openshift-service-ca/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018503083Z namespaces/openshift-service-ca/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018534564Z namespaces/openshift-service-ca/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018583606Z namespaces/openshift-service-ca/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018626667Z namespaces/openshift-service-ca/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018672468Z namespaces/openshift-service-ca/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018681048Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018707979Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.018879334Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019048339Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019188763Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019398909Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019580244Z namespaces/openshift-service-ca/pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019589134Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019622925Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/service-ca-758f599c5-4gjh9.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019697648Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/service-ca-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019707388Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/service-ca-controller/service-ca-controller/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019711218Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/service-ca-controller/service-ca-controller/logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.019746899Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/service-ca-controller/service-ca-controller/logs/current.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020010526Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/service-ca-controller/service-ca-controller/logs/previous.insecure.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020067528Z namespaces/openshift-service-ca/pods/service-ca-758f599c5-4gjh9/service-ca-controller/service-ca-controller/logs/previous.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020086819Z namespaces/openshift-service-ca/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02014216Z namespaces/openshift-service-ca/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020167351Z namespaces/openshift-service-ca/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020230303Z namespaces/openshift-service-ca/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020273764Z namespaces/openshift-user-workload-monitoring/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020304815Z namespaces/openshift-user-workload-monitoring/openshift-user-workload-monitoring.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020368747Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020412858Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02046139Z namespaces/openshift-user-workload-monitoring/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020499111Z namespaces/openshift-user-workload-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020580253Z namespaces/openshift-user-workload-monitoring/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020661375Z namespaces/openshift-user-workload-monitoring/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020740407Z namespaces/openshift-user-workload-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020774608Z namespaces/openshift-user-workload-monitoring/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020855781Z namespaces/openshift-user-workload-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020906582Z namespaces/openshift-user-workload-monitoring/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.020948913Z namespaces/openshift-user-workload-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021030546Z namespaces/openshift-user-workload-monitoring/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021071357Z namespaces/openshift-user-workload-monitoring/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021114608Z namespaces/openshift-user-workload-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021196491Z namespaces/openshift-user-workload-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021240192Z namespaces/openshift-user-workload-monitoring/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021273223Z namespaces/openshift-user-workload-monitoring/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021378366Z namespaces/openshift-user-workload-monitoring/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021456818Z namespaces/openshift-user-workload-monitoring/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02153641Z namespaces/openshift-user-workload-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021618213Z namespaces/openshift-user-workload-monitoring/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.021696135Z namespaces/openshift-user-workload-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.0218673Z namespaces/openshift-user-workload-monitoring/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022072006Z namespaces/openshift-user-workload-monitoring/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022117637Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022173888Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02221649Z namespaces/openshift-user-workload-monitoring/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022262261Z namespaces/openshift-user-workload-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022303002Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022344484Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022423056Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022502468Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022546979Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02258554Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022618381Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022674523Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022715404Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022722554Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.022759015Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02292318Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023096265Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023231349Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023437025Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02360995Z namespaces/openshift-user-workload-monitoring/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023643391Z namespaces/openshift-user-workload-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023696412Z namespaces/openshift-user-workload-monitoring/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023742093Z namespaces/openshift-user-workload-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023779625Z namespaces/openshift/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023851767Z namespaces/openshift/openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023899568Z namespaces/openshift/apps.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02394773Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.023989701Z namespaces/openshift/apps/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024033132Z namespaces/openshift/apps/daemonsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024116584Z namespaces/openshift/apps/deployments.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024203577Z namespaces/openshift/apps/replicasets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024279719Z namespaces/openshift/apps/statefulsets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024350111Z namespaces/openshift/autoscaling/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024393022Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024440714Z namespaces/openshift/batch/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024482155Z namespaces/openshift/batch/cronjobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024563067Z namespaces/openshift/batch/jobs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024605628Z namespaces/openshift/build.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02464918Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024736942Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024769093Z namespaces/openshift/core/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024833365Z namespaces/openshift/core/configmaps.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.024934648Z namespaces/openshift/core/endpoints.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02501578Z namespaces/openshift/core/events.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025097172Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025174015Z namespaces/openshift/core/pods.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025261937Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025406831Z namespaces/openshift/core/secrets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025611547Z namespaces/openshift/core/services.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025655308Z namespaces/openshift/discovery.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025685939Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025730701Z namespaces/openshift/image.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.025781762Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027156731Z namespaces/openshift/image.openshift.io/imagestreams/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027187742Z namespaces/openshift/image.openshift.io/imagestreams/cli-artifacts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027301495Z namespaces/openshift/image.openshift.io/imagestreams/cli.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027390738Z namespaces/openshift/image.openshift.io/imagestreams/dotnet-runtime.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027492421Z namespaces/openshift/image.openshift.io/imagestreams/dotnet.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027596704Z namespaces/openshift/image.openshift.io/imagestreams/driver-toolkit.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027680897Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift-java11.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.027897213Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.028063308Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.028186771Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java11-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.028284774Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift-jdk11.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.028371926Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02849315Z namespaces/openshift/image.openshift.io/imagestreams/golang.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.028597643Z namespaces/openshift/image.openshift.io/imagestreams/httpd.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.028732256Z namespaces/openshift/image.openshift.io/imagestreams/installer-artifacts.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02885046Z namespaces/openshift/image.openshift.io/imagestreams/installer.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.028960533Z namespaces/openshift/image.openshift.io/imagestreams/java-runtime.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029046766Z namespaces/openshift/image.openshift.io/imagestreams/java.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029150629Z namespaces/openshift/image.openshift.io/imagestreams/jboss-datagrid73-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029275182Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029369245Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029471528Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.02955872Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029657603Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029783307Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029873809Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.029970332Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-runtime-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.030055705Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk11-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.030168918Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk8-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.030276351Z namespaces/openshift/image.openshift.io/imagestreams/jenkins-agent-base.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.030369494Z namespaces/openshift/image.openshift.io/imagestreams/jenkins.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.030498367Z namespaces/openshift/image.openshift.io/imagestreams/mariadb.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03060644Z namespaces/openshift/image.openshift.io/imagestreams/must-gather.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.030690873Z namespaces/openshift/image.openshift.io/imagestreams/mysql.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.030818867Z namespaces/openshift/image.openshift.io/imagestreams/network-tools.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03093255Z namespaces/openshift/image.openshift.io/imagestreams/nginx.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031035143Z namespaces/openshift/image.openshift.io/imagestreams/nodejs.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031185107Z namespaces/openshift/image.openshift.io/imagestreams/oauth-proxy.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03128263Z namespaces/openshift/image.openshift.io/imagestreams/openjdk-11-rhel7.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031404683Z namespaces/openshift/image.openshift.io/imagestreams/perl.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031521857Z namespaces/openshift/image.openshift.io/imagestreams/php.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031624599Z namespaces/openshift/image.openshift.io/imagestreams/postgresql.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031753413Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso75-openshift-rhel8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031865886Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso76-openshift-rhel8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.031963839Z namespaces/openshift/image.openshift.io/imagestreams/python.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032082993Z namespaces/openshift/image.openshift.io/imagestreams/redhat-openjdk18-openshift.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032207696Z namespaces/openshift/image.openshift.io/imagestreams/redis.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032309819Z namespaces/openshift/image.openshift.io/imagestreams/ruby.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032420742Z namespaces/openshift/image.openshift.io/imagestreams/sso75-openshift-rhel8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032508715Z namespaces/openshift/image.openshift.io/imagestreams/sso76-openshift-rhel8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032606148Z namespaces/openshift/image.openshift.io/imagestreams/tests.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03269286Z namespaces/openshift/image.openshift.io/imagestreams/tools.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032828904Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11-runtime.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.032965028Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033071201Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17-runtime.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033187694Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033300818Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21-runtime.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033409951Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033486313Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8-runtime.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033611796Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033688909Z namespaces/openshift/k8s.ovn.org/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03372591Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033830043Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033918075Z namespaces/openshift/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033950316Z namespaces/openshift/monitoring.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.033994027Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034040309Z namespaces/openshift/networking.k8s.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03409111Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034125521Z namespaces/openshift/operators.coreos.com/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034133911Z namespaces/openshift/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034190603Z namespaces/openshift/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034328027Z namespaces/openshift/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034506572Z namespaces/openshift/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034634016Z namespaces/openshift/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.034887683Z namespaces/openshift/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035179172Z namespaces/openshift/policy/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035217423Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035262804Z namespaces/openshift/route.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035309085Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035362987Z namespaces/openshift/template.openshift.io/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035371617Z namespaces/openshift/template.openshift.io/templates/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035414168Z namespaces/openshift/template.openshift.io/templates/cache-service.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035514691Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035631054Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035736628Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.035870061Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.036066747Z namespaces/openshift/template.openshift.io/templates/datagrid-service.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03617189Z namespaces/openshift/template.openshift.io/templates/django-psql-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.036287233Z namespaces/openshift/template.openshift.io/templates/django-psql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.036393426Z namespaces/openshift/template.openshift.io/templates/eap-xp3-basic-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.0365176Z namespaces/openshift/template.openshift.io/templates/eap-xp4-basic-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.036628073Z namespaces/openshift/template.openshift.io/templates/eap74-basic-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.036738896Z namespaces/openshift/template.openshift.io/templates/eap74-https-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037011994Z namespaces/openshift/template.openshift.io/templates/eap74-sso-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037154688Z namespaces/openshift/template.openshift.io/templates/httpd-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037254121Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral-monitored.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037375464Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037482628Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent-monitored.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037619871Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037714804Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037817447Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.037951341Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.038041714Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.038149057Z namespaces/openshift/template.openshift.io/templates/mariadb-ephemeral.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.038300461Z namespaces/openshift/template.openshift.io/templates/mariadb-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.038404814Z namespaces/openshift/template.openshift.io/templates/mysql-ephemeral.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.038504917Z namespaces/openshift/template.openshift.io/templates/mysql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03860528Z namespaces/openshift/template.openshift.io/templates/nginx-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.038690932Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.038816456Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.03894654Z namespaces/openshift/template.openshift.io/templates/openjdk-web-basic-s2i.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039051263Z namespaces/openshift/template.openshift.io/templates/postgresql-ephemeral.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039152195Z namespaces/openshift/template.openshift.io/templates/postgresql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039252718Z namespaces/openshift/template.openshift.io/templates/rails-pgsql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039389462Z namespaces/openshift/template.openshift.io/templates/rails-postgresql-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039503516Z namespaces/openshift/template.openshift.io/templates/react-web-app-example.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039611739Z namespaces/openshift/template.openshift.io/templates/redis-ephemeral.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039712231Z namespaces/openshift/template.openshift.io/templates/redis-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039844225Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-rest-3scale.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.039954838Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-xml.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040061592Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040156974Z namespaces/openshift/template.openshift.io/templates/sso75-https.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040280038Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-https.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040389631Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040508184Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040631798Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040761452Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-https.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.040924486Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041133292Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041257276Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-https.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041370319Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041478072Z network_logs/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041506813Z network_logs/cluster_scale [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041599265Z network_logs/ippools.whereabouts.cni.cncf.io [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041673348Z network_logs/multi-networkpolicy [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.04175475Z network_logs/net-attach-def [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.041873103Z network_logs/overlappingrangeipreservations.whereabouts.cni.cncf.io [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.042010197Z network_logs/ovn_kubernetes_top_pods [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.04209897Z network_logs/ovnk_database_store.tar.gz [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.044459408Z nodes/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.044487228Z nodes/debug [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.044595331Z nodes/ip-10-0-129-204.ec2.internal/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.044637533Z nodes/ip-10-0-129-204.ec2.internal/cpu_affinities.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.045007683Z nodes/ip-10-0-129-204.ec2.internal/dmesg [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.045306512Z nodes/ip-10-0-129-204.ec2.internal/ethtool_channels [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.045381464Z nodes/ip-10-0-129-204.ec2.internal/ethtool_features [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.045487787Z nodes/ip-10-0-129-204.ec2.internal/ip-10-0-129-204.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.046031763Z nodes/ip-10-0-129-204.ec2.internal/irq_affinities.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.046181417Z nodes/ip-10-0-129-204.ec2.internal/lscpu [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.04630616Z nodes/ip-10-0-129-204.ec2.internal/lspci [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.046395873Z nodes/ip-10-0-129-204.ec2.internal/podresources.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.046488706Z nodes/ip-10-0-129-204.ec2.internal/pods_info.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.04661981Z nodes/ip-10-0-129-204.ec2.internal/proc_cmdline [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.046690752Z nodes/ip-10-0-129-204.ec2.internal/sysinfo.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.048416351Z nodes/ip-10-0-129-204.ec2.internal/sysinfo.tgz [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.048568996Z nodes/ip-10-0-140-244.ec2.internal/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.048607177Z nodes/ip-10-0-140-244.ec2.internal/cpu_affinities.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.048929336Z nodes/ip-10-0-140-244.ec2.internal/dmesg [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.049198034Z nodes/ip-10-0-140-244.ec2.internal/ethtool_channels [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.049270566Z nodes/ip-10-0-140-244.ec2.internal/ethtool_features [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.049356998Z nodes/ip-10-0-140-244.ec2.internal/ip-10-0-140-244.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.049715688Z nodes/ip-10-0-140-244.ec2.internal/irq_affinities.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.049919144Z nodes/ip-10-0-140-244.ec2.internal/lscpu [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.050069768Z nodes/ip-10-0-140-244.ec2.internal/lspci [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.050145971Z nodes/ip-10-0-140-244.ec2.internal/podresources.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.050241004Z nodes/ip-10-0-140-244.ec2.internal/pods_info.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.050347367Z nodes/ip-10-0-140-244.ec2.internal/proc_cmdline [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.050451919Z nodes/ip-10-0-140-244.ec2.internal/sysinfo.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.052153738Z nodes/ip-10-0-140-244.ec2.internal/sysinfo.tgz [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.052314053Z nodes/ip-10-0-142-38.ec2.internal/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.052340713Z nodes/ip-10-0-142-38.ec2.internal/cpu_affinities.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.052632692Z nodes/ip-10-0-142-38.ec2.internal/dmesg [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.052995662Z nodes/ip-10-0-142-38.ec2.internal/ethtool_channels [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.053065584Z nodes/ip-10-0-142-38.ec2.internal/ethtool_features [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.053156687Z nodes/ip-10-0-142-38.ec2.internal/ip-10-0-142-38.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.053566979Z nodes/ip-10-0-142-38.ec2.internal/irq_affinities.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.053704873Z nodes/ip-10-0-142-38.ec2.internal/lscpu [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.053844227Z nodes/ip-10-0-142-38.ec2.internal/lspci [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.053931909Z nodes/ip-10-0-142-38.ec2.internal/podresources.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.054037802Z nodes/ip-10-0-142-38.ec2.internal/pods_info.json [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.054154646Z nodes/ip-10-0-142-38.ec2.internal/proc_cmdline [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.054239578Z nodes/ip-10-0-142-38.ec2.internal/sysinfo.log [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.055937667Z nodes/ip-10-0-142-38.ec2.internal/sysinfo.tgz [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.056104261Z pod_network_connectivity_check/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.056133142Z pod_network_connectivity_check/podnetworkconnectivitychecks.yaml [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.056242005Z static-pods/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.056251216Z static-pods/kube-apiserver/ [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.063068271Z [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.063096022Z sent 71,442 bytes received 8,413,792 bytes 5,656,822.67 bytes/sec [must-gather] [must-gather-2sn4t] OUT 2026-07-08T17:55:34.063114433Z total size is 96,130,103 speedup is 11.33 [must-gather] [must-gather ] OUT 2026-07-08T17:55:34.399373425Z namespace/openshift-must-gather-h4rcg deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: d1dd22ee-2e29-42b5-ba81-0a2594b31f7b [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.23" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [git-push-artifacts] WORK_DIR: /workspace/odh-ci-artifacts [git-push-artifacts] REPO_PATH: opendatahub-io/odh-build-metadata [git-push-artifacts] REPO_BRANCH: ci-artifacts [git-push-artifacts] SPARSE_FILE_PATH: test-artifacts/docs [git-push-artifacts] SOURCE_PATH: /workspace/artifacts-dir [git-push-artifacts] DEST_PATH: test-artifacts/kserve-group-test-pwdft [git-push-artifacts] ALWAYS_PASS: false [git-push-artifacts] configuring gh token [git-push-artifacts] taking github token from Konflux bot [git-push-artifacts] Initialized empty Git repository in /workspace/odh-ci-artifacts/.git/ [git-push-artifacts] Using partial fetch with sparse checkout for: test-artifacts/docs [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] * [new branch] ci-artifacts -> origin/ci-artifacts [git-push-artifacts] Already on 'ci-artifacts' [git-push-artifacts] branch 'ci-artifacts' set up to track 'origin/ci-artifacts'. [git-push-artifacts] TASK_NAME=kserve-group-test-pwdft-e2e-llm-inference-service [git-push-artifacts] PIPELINERUN_NAME=kserve-group-test-pwdft [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] -rw-r--r--. 1 root 1001540000 15978237 Jul 8 17:57 /workspace/odh-ci-artifacts/test-artifacts/kserve-group-test-pwdft/e2e-llm-inference-service.tar.gz [git-push-artifacts] [ci-artifacts a8df07a] Updating CI Artifacts in e2e-llm-inference-service [git-push-artifacts] 1 file changed, 0 insertions(+), 0 deletions(-) [git-push-artifacts] create mode 100644 test-artifacts/kserve-group-test-pwdft/e2e-llm-inference-service.tar.gz [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] To https://github.com/opendatahub-io/odh-build-metadata.git [git-push-artifacts] ! [remote rejected] ci-artifacts -> ci-artifacts (cannot lock ref 'refs/heads/ci-artifacts': is at 5cef841a9c8925ead8744e3fe07048665251fbff but expected d377017ea46eb4724008157c954f69ed2d588f4a) [git-push-artifacts] error: failed to push some refs to 'https://github.com/opendatahub-io/odh-build-metadata.git' [fail-if-needed] Failing pipeline because deploy-and-e2e step failed container step-fail-if-needed has failed : [{"key":"StartedAt","value":"2026-07-08T17:57:28.007Z","type":3}]