task e2e-llm-inference-service has failed: "step-fail-if-needed" exited with code 1: Error [get-kubeconfig] Found kubeconfig secret: cluster-xjjrz-admin-kubeconfig [get-kubeconfig] Wrote kubeconfig to /credentials/cluster-xjjrz-kubeconfig [get-kubeconfig] Found admin password secret: cluster-xjjrz-admin-password [get-kubeconfig] Retrieved username [get-kubeconfig] Wrote password to /credentials/cluster-xjjrz-password [get-kubeconfig] API Server URL: https://a058b0e053a364039a705a4c8606d4df-530244ab7b089203.elb.us-east-1.amazonaws.com:6443 [get-kubeconfig] Console URL: https://console-openshift-console.apps.32f21fc7-bfe9-438c-880b-7e8aaa8ec2d9.prod.konfluxeaas.com [clone-repo] fix/ci/xks-tls-timeout [clone-repo] https://github.com/bartoszmajsak/kserve [clone-repo] Cloning into '/workspace/source'... [clone-repo] Updating files: 89% (2869/3222) Updating files: 90% (2900/3222) Updating files: 91% (2933/3222) Updating files: 92% (2965/3222) Updating files: 93% (2997/3222) Updating files: 94% (3029/3222) Updating files: 95% (3061/3222) Updating files: 96% (3094/3222) Updating files: 97% (3126/3222) Updating files: 98% (3158/3222) Updating files: 99% (3190/3222) Updating files: 100% (3222/3222) Updating files: 100% (3222/3222), done. [e2e-llm-inference-service] + bash [e2e-llm-inference-service] + STATUS_FILE=/test-status/deploy-and-e2e-status [e2e-llm-inference-service] + echo failed [e2e-llm-inference-service] + COMPONENT_NAME=kserve-agent-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-agent-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:07d9be4ee48ab8e3f13b5e400054db7129a4dfd20edc4167e0ae24bd38c38bfa [e2e-llm-inference-service] + KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:07d9be4ee48ab8e3f13b5e400054db7129a4dfd20edc4167e0ae24bd38c38bfa [e2e-llm-inference-service] + COMPONENT_NAME=kserve-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:3408a56bab73a176a3dc16d9a410a129dabf9c66a9bc9e7717bfc1996a904598 [e2e-llm-inference-service] + KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:3408a56bab73a176a3dc16d9a410a129dabf9c66a9bc9e7717bfc1996a904598 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-router-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-router-ci '.[$component_name].image' [e2e-llm-inference-service] + export KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:ee73d7a22df8b79aa8a5e917cb435b1d2b100d4582439c437bdc01a1d2422290 [e2e-llm-inference-service] + KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:ee73d7a22df8b79aa8a5e917cb435b1d2b100d4582439c437bdc01a1d2422290 [e2e-llm-inference-service] + COMPONENT_NAME=kserve-storage-initializer-ci [e2e-llm-inference-service] ++ jq -r --arg component_name kserve-storage-initializer-ci '.[$component_name].image' [e2e-llm-inference-service] + export STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] + STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] + COMPONENT_NAME=odh-kserve-llmisvc-controller-ci [e2e-llm-inference-service] ++ jq -r --arg component_name odh-kserve-llmisvc-controller-ci '.[$component_name].image' [e2e-llm-inference-service] + export LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0dc021ec33e3c2d04be2aaadffc003a21b60f42be583c38888b13b726ab7c5d0 [e2e-llm-inference-service] + LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0dc021ec33e3c2d04be2aaadffc003a21b60f42be583c38888b13b726ab7c5d0 [e2e-llm-inference-service] + ./test/scripts/openshift-ci/run-e2e-tests.sh 'llminferenceservice and cluster_cpu and not autoscaling and not tracing' 2 llm-d [e2e-llm-inference-service] INFERENCE_POOL_GROUP=inference.networking.k8s.io (detected from OCP 4.21.25) [e2e-llm-inference-service] Installing on cluster [e2e-llm-inference-service] Using namespace: kserve for KServe components [e2e-llm-inference-service] SKLEARN_IMAGE=quay.io/opendatahub/sklearn-serving-runtime:odh-pr-1771 [e2e-llm-inference-service] OPT_125M_MODEL_URI=s3://example-models/facebook/opt-125m [e2e-llm-inference-service] ERROR_404_ISVC_IMAGE=quay.io/opendatahub/error-404-isvc:odh-pr-1771 [e2e-llm-inference-service] SUCCESS_200_ISVC_IMAGE=quay.io/opendatahub/success-200-isvc:odh-pr-1771 [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed Kustomize v5.8.1 to /workspace/source/bin/kustomize [e2e-llm-inference-service] v5.8.1 [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] [INFO] Installing yq v4.52.1 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed yq v4.52.1 to /workspace/source/bin/yq [e2e-llm-inference-service] yq (https://github.com/mikefarah/yq/) version v4.52.1 [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Installing KServe Python SDK ... [e2e-llm-inference-service] [INFO] Installing uv 0.7.8 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed uv 0.7.8 to /workspace/source/bin/uv [e2e-llm-inference-service] warning: Failed to read project metadata (No `pyproject.toml` found in current directory or any parent directory). Running `uv self version` for compatibility. This fallback will be removed in the future; pass `--preview` to force an error. [e2e-llm-inference-service] uv 0.7.8 [e2e-llm-inference-service] Creating virtual environment... [e2e-llm-inference-service] warning: virtualenv's `--clear` has no effect (uv always clears the virtual environment) [e2e-llm-inference-service] Using CPython 3.9.25 interpreter at: /usr/bin/python3 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] Using CPython 3.11.13 interpreter at: /usr/bin/python3.11 [e2e-llm-inference-service] Creating virtual environment at: .venv [e2e-llm-inference-service] Resolved 274 packages in 1ms [e2e-llm-inference-service] Building kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading cryptography (4.5MiB) [e2e-llm-inference-service] Downloading aiohttp (1.7MiB) [e2e-llm-inference-service] Downloading pydantic-core (2.0MiB) [e2e-llm-inference-service] Downloading uvloop (3.8MiB) [e2e-llm-inference-service] Downloading numpy (15.7MiB) [e2e-llm-inference-service] Downloading kubernetes (1.9MiB) [e2e-llm-inference-service] Downloading portforward (3.9MiB) [e2e-llm-inference-service] Downloading botocore (12.9MiB) [e2e-llm-inference-service] Downloading pandas (12.5MiB) [e2e-llm-inference-service] Downloading mypy (17.2MiB) [e2e-llm-inference-service] Downloading pyarrow (40.1MiB) [e2e-llm-inference-service] Downloading setuptools (1.2MiB) [e2e-llm-inference-service] Downloading grpcio (6.4MiB) [e2e-llm-inference-service] Downloading black (1.6MiB) [e2e-llm-inference-service] Downloading grpcio-tools (2.5MiB) [e2e-llm-inference-service] Building timeout-sampler==1.0.3 [e2e-llm-inference-service] Building python-simple-logger==2.0.19 [e2e-llm-inference-service] Downloading aiohttp [e2e-llm-inference-service] Downloading black [e2e-llm-inference-service] Downloading pydantic-core [e2e-llm-inference-service] Downloading grpcio-tools [e2e-llm-inference-service] Downloading setuptools [e2e-llm-inference-service] Built python-simple-logger==2.0.19 [e2e-llm-inference-service] Downloading portforward [e2e-llm-inference-service] Downloading uvloop [e2e-llm-inference-service] Downloading cryptography [e2e-llm-inference-service] Downloading grpcio [e2e-llm-inference-service] Downloading kubernetes [e2e-llm-inference-service] Built timeout-sampler==1.0.3 [e2e-llm-inference-service] Downloading numpy [e2e-llm-inference-service] Built kserve @ file:///workspace/source/python/kserve [e2e-llm-inference-service] Downloading pandas [e2e-llm-inference-service] Downloading botocore [e2e-llm-inference-service] Downloading pyarrow [e2e-llm-inference-service] Downloading mypy [e2e-llm-inference-service] Prepared 101 packages in 1.89s [e2e-llm-inference-service] warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance. [e2e-llm-inference-service] If the cache and target directories are on different filesystems, hardlinking may not be supported. [e2e-llm-inference-service] If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning. [e2e-llm-inference-service] Installed 101 packages in 336ms [e2e-llm-inference-service] + aiohappyeyeballs==2.6.1 [e2e-llm-inference-service] + aiohttp==3.14.1 [e2e-llm-inference-service] + aiosignal==1.4.0 [e2e-llm-inference-service] + annotated-doc==0.0.4 [e2e-llm-inference-service] + annotated-types==0.7.0 [e2e-llm-inference-service] + anyio==4.9.0 [e2e-llm-inference-service] + attrs==25.3.0 [e2e-llm-inference-service] + avro==1.12.0 [e2e-llm-inference-service] + black==24.3.0 [e2e-llm-inference-service] + boto3==1.37.35 [e2e-llm-inference-service] + botocore==1.37.35 [e2e-llm-inference-service] + cachetools==5.5.2 [e2e-llm-inference-service] + certifi==2025.1.31 [e2e-llm-inference-service] + cffi==2.0.0 [e2e-llm-inference-service] + charset-normalizer==3.4.1 [e2e-llm-inference-service] + click==8.4.2 [e2e-llm-inference-service] + cloudevents==1.11.0 [e2e-llm-inference-service] + colorama==0.4.6 [e2e-llm-inference-service] + colorlog==6.10.1 [e2e-llm-inference-service] + coverage==7.8.0 [e2e-llm-inference-service] + cryptography==49.0.0 [e2e-llm-inference-service] + deprecation==2.1.0 [e2e-llm-inference-service] + durationpy==0.9 [e2e-llm-inference-service] + execnet==2.1.1 [e2e-llm-inference-service] + fastapi==0.136.3 [e2e-llm-inference-service] + frozenlist==1.5.0 [e2e-llm-inference-service] + google-auth==2.39.0 [e2e-llm-inference-service] + grpc-interceptor==0.15.4 [e2e-llm-inference-service] + grpcio==1.78.1 [e2e-llm-inference-service] + grpcio-testing==1.78.1 [e2e-llm-inference-service] + grpcio-tools==1.78.1 [e2e-llm-inference-service] + h11==0.16.0 [e2e-llm-inference-service] + httpcore==1.0.9 [e2e-llm-inference-service] + httptools==0.6.4 [e2e-llm-inference-service] + httpx==0.27.2 [e2e-llm-inference-service] + httpx-retries==0.4.5 [e2e-llm-inference-service] + idna==3.10 [e2e-llm-inference-service] + iniconfig==2.1.0 [e2e-llm-inference-service] + jinja2==3.1.6 [e2e-llm-inference-service] + jmespath==1.0.1 [e2e-llm-inference-service] + kserve==0.20.0rc0 (from file:///workspace/source/python/kserve) [e2e-llm-inference-service] + kubernetes==32.0.1 [e2e-llm-inference-service] + markupsafe==3.0.2 [e2e-llm-inference-service] + multidict==6.4.3 [e2e-llm-inference-service] + mypy==0.991 [e2e-llm-inference-service] + mypy-extensions==1.0.0 [e2e-llm-inference-service] + numpy==2.2.4 [e2e-llm-inference-service] + oauthlib==3.2.2 [e2e-llm-inference-service] + orjson==3.11.9 [e2e-llm-inference-service] + packaging==24.2 [e2e-llm-inference-service] + pandas==2.2.3 [e2e-llm-inference-service] + pathspec==0.12.1 [e2e-llm-inference-service] + platformdirs==4.3.7 [e2e-llm-inference-service] + pluggy==1.5.0 [e2e-llm-inference-service] + portforward==0.7.1 [e2e-llm-inference-service] + prometheus-client==0.21.1 [e2e-llm-inference-service] + propcache==0.3.1 [e2e-llm-inference-service] + protobuf==6.33.5 [e2e-llm-inference-service] + psutil==5.9.8 [e2e-llm-inference-service] + pyarrow==19.0.1 [e2e-llm-inference-service] + pyasn1==0.6.3 [e2e-llm-inference-service] + pyasn1-modules==0.4.2 [e2e-llm-inference-service] + pycparser==2.22 [e2e-llm-inference-service] + pydantic==2.12.4 [e2e-llm-inference-service] + pydantic-core==2.41.5 [e2e-llm-inference-service] + pyjwt==2.13.0 [e2e-llm-inference-service] + pytest==7.4.4 [e2e-llm-inference-service] + pytest-asyncio==0.23.8 [e2e-llm-inference-service] + pytest-cov==5.0.0 [e2e-llm-inference-service] + pytest-httpx==0.30.0 [e2e-llm-inference-service] + pytest-json-report==1.5.0 [e2e-llm-inference-service] + pytest-metadata==3.1.1 [e2e-llm-inference-service] + pytest-xdist==3.6.1 [e2e-llm-inference-service] + python-dateutil==2.9.0.post0 [e2e-llm-inference-service] + python-dotenv==1.1.0 [e2e-llm-inference-service] + python-multipart==0.0.32 [e2e-llm-inference-service] + python-simple-logger==2.0.19 [e2e-llm-inference-service] + pytz==2025.2 [e2e-llm-inference-service] + pyyaml==6.0.2 [e2e-llm-inference-service] + requests==2.32.3 [e2e-llm-inference-service] + requests-oauthlib==2.0.0 [e2e-llm-inference-service] + rsa==4.9.1 [e2e-llm-inference-service] + s3transfer==0.11.4 [e2e-llm-inference-service] + setuptools==78.1.0 [e2e-llm-inference-service] + six==1.17.0 [e2e-llm-inference-service] + sniffio==1.3.1 [e2e-llm-inference-service] + starlette==1.2.1 [e2e-llm-inference-service] + tabulate==0.9.0 [e2e-llm-inference-service] + timeout-sampler==1.0.3 [e2e-llm-inference-service] + timing-asgi==0.3.1 [e2e-llm-inference-service] + tomlkit==0.13.2 [e2e-llm-inference-service] + typing-extensions==4.15.0 [e2e-llm-inference-service] + typing-inspection==0.4.2 [e2e-llm-inference-service] + tzdata==2025.2 [e2e-llm-inference-service] + urllib3==2.7.0 [e2e-llm-inference-service] + uvicorn==0.34.1 [e2e-llm-inference-service] + uvloop==0.21.0 [e2e-llm-inference-service] + watchfiles==1.0.5 [e2e-llm-inference-service] + websocket-client==1.8.0 [e2e-llm-inference-service] + websockets==15.0.1 [e2e-llm-inference-service] + yarl==1.20.0 [e2e-llm-inference-service] Audited 1 package in 49ms [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [INFO] Kustomize v5.8.1 is already installed in /workspace/source/bin (>= v5.8.1) [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Now using project "kserve" on server "https://a058b0e053a364039a705a4c8606d4df-530244ab7b089203.elb.us-east-1.amazonaws.com:6443". [e2e-llm-inference-service] [e2e-llm-inference-service] You can add applications to this project with the 'new-app' command. For example, try: [e2e-llm-inference-service] [e2e-llm-inference-service] oc new-app rails-postgresql-example [e2e-llm-inference-service] [e2e-llm-inference-service] to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application: [e2e-llm-inference-service] [e2e-llm-inference-service] kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname [e2e-llm-inference-service] [e2e-llm-inference-service] [INFO] Installing Kustomize v5.8.1 for linux/amd64... [e2e-llm-inference-service] [INFO] Kustomize v5.8.1 is already installed in /workspace/source/bin (>= v5.8.1) [e2e-llm-inference-service] make: Entering directory '/workspace/source' [e2e-llm-inference-service] make: Leaving directory '/workspace/source' [e2e-llm-inference-service] Creating namespace openshift-keda... [e2e-llm-inference-service] namespace/openshift-keda created [e2e-llm-inference-service] Namespace openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating OperatorGroup openshift-keda... [e2e-llm-inference-service] operatorgroup.operators.coreos.com/openshift-keda created [e2e-llm-inference-service] OperatorGroup openshift-keda created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Creating Subscription for openshift-custom-metrics-autoscaler-operator... [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-custom-metrics-autoscaler-operator created [e2e-llm-inference-service] Subscription openshift-custom-metrics-autoscaler-operator created/ensured. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Waiting for openshift-custom-metrics-autoscaler-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (15/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-custom-metrics-autoscaler-operator... (20/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (25/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (40/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 found, but not yet Succeeded (Phase: Installing). Waiting... (45/600) [e2e-llm-inference-service] CSV custom-metrics-autoscaler.v2.19.0-2 is ready (Phase: Succeeded). [e2e-llm-inference-service] --- [e2e-llm-inference-service] Applying KedaController custom resource... [e2e-llm-inference-service] Warning: resource kedacontrollers/keda is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] kedacontroller.keda.sh/keda configured [e2e-llm-inference-service] KedaController custom resource applied. [e2e-llm-inference-service] --- [e2e-llm-inference-service] Allowing time for KEDA components to be provisioned by the operator ... [e2e-llm-inference-service] Waiting for KEDA Operator pod (selector: "app=keda-operator") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-operator" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-operator" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-operator" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-operator-77f648d694-kd6w2 1/1 Running 0 43s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-operator" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-operator-77f648d694-kd6w2 condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-operator" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Operator pod is ready. [e2e-llm-inference-service] Waiting for KEDA Metrics API Server pod (selector: "app=keda-metrics-apiserver") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-metrics-apiserver" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-metrics-apiserver" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-metrics-apiserver-55557df8c-wt86z 1/1 Running 0 49s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-metrics-apiserver-55557df8c-wt86z condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-metrics-apiserver" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Metrics API Server pod is ready. [e2e-llm-inference-service] Waiting for KEDA Webhook pod (selector: "app=keda-admission-webhooks") to be ready in namespace openshift-keda... [e2e-llm-inference-service] Waiting for pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" to be created... [e2e-llm-inference-service] Pod -l "app=keda-admission-webhooks" in namespace "openshift-keda" found. [e2e-llm-inference-service] Current pods for -l "app=keda-admission-webhooks" in namespace "openshift-keda": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] keda-admission-56f9798bb4-v9gzk 1/1 Running 0 53s [e2e-llm-inference-service] Waiting up to 120s for pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" to become ready... [e2e-llm-inference-service] pod/keda-admission-56f9798bb4-v9gzk condition met [e2e-llm-inference-service] Pod(s) -l "app=keda-admission-webhooks" in namespace "openshift-keda" are ready. [e2e-llm-inference-service] KEDA Webhook pod is ready. [e2e-llm-inference-service] --- [e2e-llm-inference-service] ✅ KEDA deployment script finished successfully. [e2e-llm-inference-service] KSERVE_CONTROLLER_IMAGE=quay.io/opendatahub/kserve-controller@sha256:3408a56bab73a176a3dc16d9a410a129dabf9c66a9bc9e7717bfc1996a904598 [e2e-llm-inference-service] LLMISVC_CONTROLLER_IMAGE=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0dc021ec33e3c2d04be2aaadffc003a21b60f42be583c38888b13b726ab7c5d0 [e2e-llm-inference-service] KSERVE_AGENT_IMAGE=quay.io/opendatahub/kserve-agent@sha256:07d9be4ee48ab8e3f13b5e400054db7129a4dfd20edc4167e0ae24bd38c38bfa [e2e-llm-inference-service] KSERVE_ROUTER_IMAGE=quay.io/opendatahub/kserve-router@sha256:ee73d7a22df8b79aa8a5e917cb435b1d2b100d4582439c437bdc01a1d2422290 [e2e-llm-inference-service] STORAGE_INITIALIZER_IMAGE=quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] Installing KServe via kustomize... [e2e-llm-inference-service] === Final params.env [e2e-llm-inference-service] kserve-controller=quay.io/opendatahub/kserve-controller@sha256:3408a56bab73a176a3dc16d9a410a129dabf9c66a9bc9e7717bfc1996a904598 [e2e-llm-inference-service] llmisvc-controller=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0dc021ec33e3c2d04be2aaadffc003a21b60f42be583c38888b13b726ab7c5d0 [e2e-llm-inference-service] kserve-agent=quay.io/opendatahub/kserve-agent@sha256:07d9be4ee48ab8e3f13b5e400054db7129a4dfd20edc4167e0ae24bd38c38bfa [e2e-llm-inference-service] kserve-router=quay.io/opendatahub/kserve-router@sha256:ee73d7a22df8b79aa8a5e917cb435b1d2b100d4582439c437bdc01a1d2422290 [e2e-llm-inference-service] kserve-storage-initializer=quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] kserve-llm-d=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-inference-scheduler=quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] kserve-llm-d-routing-sidecar=quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0 [e2e-llm-inference-service] kserve-llm-d-uds-tokenizer=quay.io/opendatahub/llm-d-kv-cache:v0.8.0 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-1=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-2=registry.redhat.io/rhaiis/vllm-cuda-rhel9@sha256:fc68d623d1bfc36c8cb2fe4a71f19c8578cfb420ce8ce07b20a02c1ee0be0cf3 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-1-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-nvidia-cuda-fast-2-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-amd-rocm=registry.redhat.io/rhaiis/vllm-rocm-rhel9@sha256:d9a48add238cc095fa43eeee17c8c4d104de60c4dc623e0bc7f8c4b53b2b2e97 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-1=registry.redhat.io/rhaiis/vllm-rocm-rhel9@sha256:d9a48add238cc095fa43eeee17c8c4d104de60c4dc623e0bc7f8c4b53b2b2e97 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-2=registry.redhat.io/rhaiis/vllm-rocm-rhel9@sha256:d9a48add238cc095fa43eeee17c8c4d104de60c4dc623e0bc7f8c4b53b2b2e97 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-1-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-amd-rocm-fast-2-upstream-version=0.11.0+rhai5 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi=registry.redhat.io/rhaii-early-access/vllm-gaudi-rhel9:3.4.0-ea.2 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-1=registry.redhat.io/rhaii-early-access/vllm-gaudi-rhel9:3.4.0-ea.2 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-2=registry.redhat.io/rhaii-early-access/vllm-gaudi-rhel9:3.4.0-ea.2 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-upstream-version=0.16.0 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-1-upstream-version=0.16.0 [e2e-llm-inference-service] kserve-llm-d-intel-gaudi-fast-2-upstream-version=0.16.0 [e2e-llm-inference-service] kserve-llm-d-ibm-spyre=registry.redhat.io/rhaiis/vllm-spyre-rhel9@sha256:80ae3e435a5be2c1f117f36599103ab05357917dd6e37f0df6613cb3ac2c13ea [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-1=registry.redhat.io/rhaiis/vllm-spyre-rhel9@sha256:80ae3e435a5be2c1f117f36599103ab05357917dd6e37f0df6613cb3ac2c13ea [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-2=registry.redhat.io/rhaiis/vllm-spyre-rhel9@sha256:80ae3e435a5be2c1f117f36599103ab05357917dd6e37f0df6613cb3ac2c13ea [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-upstream-version=0.10.2 [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-1-upstream-version=0.10.2 [e2e-llm-inference-service] kserve-llm-d-ibm-spyre-fast-2-upstream-version=0.10.2 [e2e-llm-inference-service] # TODO update when our changes are introduced in the official image [e2e-llm-inference-service] kube-rbac-proxy=quay.io/opendatahub/odh-kube-auth-proxy@sha256:dcb09fbabd8811f0956ef612a0c9ddd5236804b9bd6548a0647d2b531c9d01b3 [e2e-llm-inference-service] kserve-localmodel-controller=quay.io/opendatahub/odh-kserve-localmodel-controller:odh-master [e2e-llm-inference-service] kserve-localmodelnode-agent=quay.io/opendatahub/odh-kserve-localmodelnode-agent:odh-master# Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] # Warning: 'commonLabels' is deprecated. Please use 'labels' instead. Run 'kustomize edit fix' to update your Kustomization automatically. [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.llm-d.ai serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.llm-d.ai serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] Waiting for CRDs to be established... [e2e-llm-inference-service] Waiting for CRD inferenceservices.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD inferenceservices.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD llminferenceserviceconfigs.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD llminferenceserviceconfigs.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD clusterstoragecontainers.serving.kserve.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD clusterstoragecontainers.serving.kserve.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io condition met [e2e-llm-inference-service] Waiting for CRD datascienceclusters.datasciencecluster.opendatahub.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD datascienceclusters.datasciencecluster.opendatahub.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io condition met [e2e-llm-inference-service] Applying all resources... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.llm-d.ai serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.llm-d.ai serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] serviceaccount/kserve-controller-manager serverside-applied [e2e-llm-inference-service] serviceaccount/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-leader-election-role serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/llmisvc-leader-election-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-inferenceservice-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-metrics-reader-cluster-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-proxy-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/llmisvc-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-inferenceservice-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-llmisvc-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/llmisvc-manager-rolebinding serverside-applied [e2e-llm-inference-service] configmap/inferenceservice-config serverside-applied [e2e-llm-inference-service] configmap/kserve-parameters serverside-applied [e2e-llm-inference-service] secret/kserve-webhook-server-secret serverside-applied [e2e-llm-inference-service] secret/mlpipeline-s3-artifact serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-metrics-service serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-service serverside-applied [e2e-llm-inference-service] service/kserve-webhook-server-service serverside-applied [e2e-llm-inference-service] service/llmisvc-controller-manager-service serverside-applied [e2e-llm-inference-service] service/llmisvc-webhook-server-service serverside-applied [e2e-llm-inference-service] service/s3-service serverside-applied [e2e-llm-inference-service] deployment.apps/kserve-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/seaweedfs serverside-applied [e2e-llm-inference-service] networkpolicy.networking.k8s.io/kserve-controller-manager serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] clusterstoragecontainer.serving.kserve.io/default serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler-latency-predictor serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-tracing serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceserviceconfig.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io serverside-applied [e2e-llm-inference-service] Waiting for llmisvc-controller-manager to be ready... [e2e-llm-inference-service] Waiting for pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to be created... [e2e-llm-inference-service] Pod -l "control-plane=llmisvc-controller-manager" in namespace "kserve" found. [e2e-llm-inference-service] Current pods for -l "control-plane=llmisvc-controller-manager" in namespace "kserve": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] llmisvc-controller-manager-5d54f9649b-7wc28 0/1 Running 0 6s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" to become ready... [e2e-llm-inference-service] pod/llmisvc-controller-manager-5d54f9649b-7wc28 condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=llmisvc-controller-manager" in namespace "kserve" are ready. [e2e-llm-inference-service] Re-Applying all resources... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencemodelrewrites.llm-d.ai serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceobjectives.llm-d.ai serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferencepools.inference.networking.x-k8s.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceserviceconfigs.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/llminferenceservices.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied [e2e-llm-inference-service] serviceaccount/kserve-controller-manager serverside-applied [e2e-llm-inference-service] serviceaccount/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-leader-election-role serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] role.rbac.authorization.k8s.io/llmisvc-leader-election-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-inferenceservice-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-distro-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-llmisvc-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-manager-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-metrics-reader-cluster-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-admin serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-edit serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-models-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-proxy-role serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-view serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/kserve-llmisvcconfig-read-access serverside-applied [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/llmisvc-leader-election-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-inferenceservice-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-llmisvc-distro-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding serverside-applied [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/llmisvc-manager-rolebinding serverside-applied [e2e-llm-inference-service] configmap/inferenceservice-config serverside-applied [e2e-llm-inference-service] configmap/kserve-parameters serverside-applied [e2e-llm-inference-service] secret/kserve-webhook-server-secret serverside-applied [e2e-llm-inference-service] secret/mlpipeline-s3-artifact serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-metrics-service serverside-applied [e2e-llm-inference-service] service/kserve-controller-manager-service serverside-applied [e2e-llm-inference-service] service/kserve-webhook-server-service serverside-applied [e2e-llm-inference-service] service/llmisvc-controller-manager-service serverside-applied [e2e-llm-inference-service] service/llmisvc-webhook-server-service serverside-applied [e2e-llm-inference-service] service/s3-service serverside-applied [e2e-llm-inference-service] deployment.apps/kserve-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/llmisvc-controller-manager serverside-applied [e2e-llm-inference-service] deployment.apps/seaweedfs serverside-applied [e2e-llm-inference-service] networkpolicy.networking.k8s.io/kserve-controller-manager serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-inferenceservice-image-volume-scc serverside-applied [e2e-llm-inference-service] securitycontextconstraints.security.openshift.io/openshift-ai-llminferenceservice-scc serverside-applied [e2e-llm-inference-service] clusterstoragecontainer.serving.kserve.io/default serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-decode-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-multi-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-prefill-worker-data-parallel serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-router-route serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-scheduler-latency-predictor serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-pd-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-single-node-template-nvidia-cuda-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-amd-rocm-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-ppc64le-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-s390x-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-ibm-spyre-x86-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-1 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-template-intel-gaudi-fast-2 serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-tracing serverside-applied [e2e-llm-inference-service] llminferenceserviceconfig.serving.kserve.io/kserve-config-llm-worker-data-parallel serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceservice.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/llminferenceserviceconfig.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io serverside-applied [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io serverside-applied [e2e-llm-inference-service] Applying DSC/DSCI resources... [e2e-llm-inference-service] dscinitialization.dscinitialization.opendatahub.io/test-dsci created [e2e-llm-inference-service] datasciencecluster.datasciencecluster.opendatahub.io/test-dsc created [e2e-llm-inference-service] KServe manual installation complete [e2e-llm-inference-service] 🔧 Configuration: [e2e-llm-inference-service] KServe deployment: ❌ disabled [e2e-llm-inference-service] Kuadrant deployment: ✅ enabled [e2e-llm-inference-service] [e2e-llm-inference-service] Checking OpenShift server version...(4.21.25) [e2e-llm-inference-service] 🎯 Server version (4.21.25) is 4.19.9 or higher - continue with the script [e2e-llm-inference-service] ⏳ Installing cert-manager [e2e-llm-inference-service] namespace/cert-manager-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/openshift-cert-manager-operator created [e2e-llm-inference-service] subscription.operators.coreos.com/openshift-cert-manager-operator created [e2e-llm-inference-service] Waiting for openshift-cert-manager-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (5/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription openshift-cert-manager-operator... (10/300) [e2e-llm-inference-service] CSV cert-manager-operator.v1.20.0 found, but not yet Succeeded (Phase: Installing). Waiting... (15/300) [e2e-llm-inference-service] CSV cert-manager-operator.v1.20.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD certificates.cert-manager.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD certificates.cert-manager.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io condition met [e2e-llm-inference-service] ✅ Cert-manager installed [e2e-llm-inference-service] ⏳ Installing openshift-lws-operator [e2e-llm-inference-service] namespace/openshift-lws-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] subscription.operators.coreos.com/leader-worker-set created [e2e-llm-inference-service] Waiting for leader-worker-set CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (0/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (5/300) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription leader-worker-set... (10/300) [e2e-llm-inference-service] CSV leader-worker-set.v1.0.0 found, but not yet Succeeded (Phase: Installing). Waiting... (15/300) [e2e-llm-inference-service] CSV leader-worker-set.v1.0.0 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD leaderworkersetoperators.operator.openshift.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD leaderworkersetoperators.operator.openshift.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/leaderworkersetoperators.operator.openshift.io condition met [e2e-llm-inference-service] leaderworkersetoperator.operator.openshift.io/cluster created [e2e-llm-inference-service] ⏳ waiting for openshift-lws-operator to be ready.… [e2e-llm-inference-service] Waiting for pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to be created... [e2e-llm-inference-service] Pod -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" found. [e2e-llm-inference-service] Current pods for -l "name=openshift-lws-operator" in namespace "openshift-lws-operator": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-lws-operator-fd8ccff4c-7msm6 1/1 Running 0 14s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" to become ready... [e2e-llm-inference-service] pod/openshift-lws-operator-fd8ccff4c-7msm6 condition met [e2e-llm-inference-service] Pod(s) -l "name=openshift-lws-operator" in namespace "openshift-lws-operator" are ready. [e2e-llm-inference-service] ✅ openshift-lws-operator installed [e2e-llm-inference-service] gatewayclass.gateway.networking.k8s.io/openshift-default created [e2e-llm-inference-service] Waiting for pod -l "app=istiod" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "app=istiod" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "app=istiod" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] istiod-openshift-gateway-94bb8fbfd-zltzb 0/1 Running 0 5s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "app=istiod" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/istiod-openshift-gateway-94bb8fbfd-zltzb condition met [e2e-llm-inference-service] Pod(s) -l "app=istiod" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Creating a Gateway [e2e-llm-inference-service] Error from server (AlreadyExists): namespaces "openshift-ingress" already exists [e2e-llm-inference-service] ⏳ Creating gateway memory ConfigMap for parametersRef (2Gi) [e2e-llm-inference-service] configmap/gateway-proxy-config created [e2e-llm-inference-service] gateway.gateway.networking.k8s.io/openshift-ai-inference created [e2e-llm-inference-service] Waiting for pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to be created... [e2e-llm-inference-service] Pod -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" found. [e2e-llm-inference-service] Current pods for -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] openshift-ai-inference-openshift-default-655f7c58b4-jfpwp 1/1 Running 0 7s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" to become ready... [e2e-llm-inference-service] pod/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp condition met [e2e-llm-inference-service] Pod(s) -l "serving.kserve.io/gateway=kserve-ingress-gateway" in namespace "openshift-ingress" are ready. [e2e-llm-inference-service] ⏳ Installing RHCL(Kuadrant) operator [e2e-llm-inference-service] namespace/kuadrant-system created [e2e-llm-inference-service] subscription.operators.coreos.com/rhcl-operator created [e2e-llm-inference-service] operatorgroup.operators.coreos.com/kuadrant created [e2e-llm-inference-service] Waiting for rhcl-operator CSV to become ready... [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (0/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (5/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (10/600) [e2e-llm-inference-service] Waiting for CSV to be installed for subscription rhcl-operator... (15/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Pending). Waiting... (20/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (25/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (30/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 found, but not yet Succeeded (Phase: Installing). Waiting... (35/600) [e2e-llm-inference-service] CSV rhcl-operator.v1.4.1 is ready (Phase: Succeeded). [e2e-llm-inference-service] Waiting for CRD kuadrants.kuadrant.io to appear (timeout: 90s)… [e2e-llm-inference-service] CRD kuadrants.kuadrant.io detected — waiting for it to become Established (timeout: 90s)… [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/kuadrants.kuadrant.io condition met [e2e-llm-inference-service] Waiting for apiserver discovery /apis/kuadrant.io/v1beta1 to list kuadrants (timeout: 120s)… [e2e-llm-inference-service] Discovery for kuadrant.io/v1beta1 includes kuadrants. [e2e-llm-inference-service] ⏳ sleeping 30s after discovery (RESTMapper can trail discovery)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant created [e2e-llm-inference-service] ⏳ waiting for Kuadrant Ready (attempt 1/2, timeout 5m)… [e2e-llm-inference-service] kuadrant.kuadrant.io/kuadrant condition met [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-6d85f6564-kxw42 1/1 Running 0 69s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-6d85f6564-kxw42 condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ⏳ waiting for authorino service to be created... [e2e-llm-inference-service] service/authorino-authorino-authorization condition met [e2e-llm-inference-service] service/authorino-authorino-authorization annotated [e2e-llm-inference-service] Warning: resource authorinos/authorino is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically. [e2e-llm-inference-service] authorino.operator.authorino.kuadrant.io/authorino configured [e2e-llm-inference-service] Waiting for pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" to be created... [e2e-llm-inference-service] Pod -l "control-plane=authorino-operator" in namespace "kuadrant-system" found. [e2e-llm-inference-service] Current pods for -l "control-plane=authorino-operator" in namespace "kuadrant-system": [e2e-llm-inference-service] NAME READY STATUS RESTARTS AGE [e2e-llm-inference-service] authorino-operator-6d85f6564-kxw42 1/1 Running 0 81s [e2e-llm-inference-service] Waiting up to 600s for pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" to become ready... [e2e-llm-inference-service] pod/authorino-operator-6d85f6564-kxw42 condition met [e2e-llm-inference-service] Pod(s) -l "control-plane=authorino-operator" in namespace "kuadrant-system" are ready. [e2e-llm-inference-service] ✅ kuadrant(authorino) installed [e2e-llm-inference-service] Setting up Jaeger (Helm) for LLMISVC tracing e2e... [e2e-llm-inference-service] ⏳ Installing Jaeger All-in-One (Helm) into namespace observability [e2e-llm-inference-service] [INFO] Installing Helm v3.16.3 for linux/amd64... [e2e-llm-inference-service] [SUCCESS] Successfully installed Helm v3.16.3 to /workspace/source/bin/helm [e2e-llm-inference-service] WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /credentials/cluster-xjjrz-kubeconfig [e2e-llm-inference-service] WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /credentials/cluster-xjjrz-kubeconfig [e2e-llm-inference-service] version.BuildInfo{Version:"v3.16.3", GitCommit:"cfd07493f46efc9debd9cc1b02a0961186df7fdf", GitTreeState:"clean", GoVersion:"go1.22.7"} [e2e-llm-inference-service] [INFO] Adding Jaeger Helm repository... [e2e-llm-inference-service] WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /credentials/cluster-xjjrz-kubeconfig [e2e-llm-inference-service] WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /credentials/cluster-xjjrz-kubeconfig [e2e-llm-inference-service] "jaegertracing" has been added to your repositories [e2e-llm-inference-service] [INFO] Installing Jaeger All-in-One 4.7.0... [e2e-llm-inference-service] WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /credentials/cluster-xjjrz-kubeconfig [e2e-llm-inference-service] WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /credentials/cluster-xjjrz-kubeconfig [e2e-llm-inference-service] NAME: jaeger [e2e-llm-inference-service] LAST DEPLOYED: Mon Jul 20 10:08:31 2026 [e2e-llm-inference-service] NAMESPACE: observability [e2e-llm-inference-service] STATUS: deployed [e2e-llm-inference-service] REVISION: 1 [e2e-llm-inference-service] TEST SUITE: None [e2e-llm-inference-service] NOTES: [e2e-llm-inference-service] ################################################################### [e2e-llm-inference-service] ### ⚠️ EXPERIMENTAL - NO STABILITY GUARANTEES ### [e2e-llm-inference-service] ### ### [e2e-llm-inference-service] ### This chart is under active development. ### [e2e-llm-inference-service] ### Breaking changes may occur in minor versions. ### [e2e-llm-inference-service] ### ### [e2e-llm-inference-service] ### See README.md for configuration details. ### [e2e-llm-inference-service] ################################################################### [e2e-llm-inference-service] [e2e-llm-inference-service] 🚀 Congratulations on successfully installing Jaeger v2.17.0 (Chart v4.7.0)! [e2e-llm-inference-service] [e2e-llm-inference-service] To access the query UI: [e2e-llm-inference-service] export POD_NAME=$(kubectl get pods --namespace observability -l "app.kubernetes.io/instance=jaeger,app.kubernetes.io/component=all-in-one" -o jsonpath="{.items[0].metadata.name}") [e2e-llm-inference-service] kubectl port-forward --namespace observability $POD_NAME 16686:16686 --address 0.0.0.0 & [e2e-llm-inference-service] Visit http://127.0.0.1:16686/ [e2e-llm-inference-service] [SUCCESS] Successfully installed Jaeger All-in-One via Helm [e2e-llm-inference-service] [INFO] Waiting for pods with label 'app.kubernetes.io/name=jaeger' in namespace 'observability' to be created... [e2e-llm-inference-service] [INFO] Found 1 pod(s) with label 'app.kubernetes.io/name=jaeger' [e2e-llm-inference-service] [INFO] Waiting for pods with label 'app.kubernetes.io/name=jaeger' in namespace 'observability' to be ready... [e2e-llm-inference-service] pod/jaeger-6bbd7d4c9d-cgnvp condition met [e2e-llm-inference-service] [SUCCESS] Pods with label 'app.kubernetes.io/name=jaeger' in namespace 'observability' are ready! [e2e-llm-inference-service] [SUCCESS] Jaeger is ready! [e2e-llm-inference-service] ⏳ Verifying Jaeger Service ports (OTLP 4317, Query 16686)... [e2e-llm-inference-service] ✅ Jaeger (Helm) ready — OTLP http://jaeger.observability.svc.cluster.local:4317 [e2e-llm-inference-service] Patching ingress domain... [e2e-llm-inference-service] configmap/inferenceservice-config patched [e2e-llm-inference-service] pod "kserve-controller-manager-645f4f9567-l9pg7" deleted [e2e-llm-inference-service] Waiting for kserve-controller-manager to be ready... [e2e-llm-inference-service] pod/kserve-controller-manager-645f4f9567-g46rm condition met [e2e-llm-inference-service] Installing ODH Model Controller manually... [e2e-llm-inference-service] customresourcedefinition.apiextensions.k8s.io/accounts.nim.opendatahub.io created [e2e-llm-inference-service] serviceaccount/model-serving-api created [e2e-llm-inference-service] serviceaccount/odh-model-controller created [e2e-llm-inference-service] role.rbac.authorization.k8s.io/leader-election-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/account-editor-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/account-viewer-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/kserve-prometheus-k8s created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/metrics-reader created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/odh-model-controller-role created [e2e-llm-inference-service] clusterrole.rbac.authorization.k8s.io/proxy-role created [e2e-llm-inference-service] rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/model-serving-api created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/odh-model-controller-rolebinding-opendatahub created [e2e-llm-inference-service] clusterrolebinding.rbac.authorization.k8s.io/proxy-rolebinding created [e2e-llm-inference-service] configmap/odh-model-controller-parameters created [e2e-llm-inference-service] service/model-serving-api created [e2e-llm-inference-service] service/odh-model-controller-metrics-service created [e2e-llm-inference-service] service/odh-model-controller-webhook-service created [e2e-llm-inference-service] deployment.apps/model-serving-api created [e2e-llm-inference-service] deployment.apps/odh-model-controller created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/model-serving-api-metrics created [e2e-llm-inference-service] servicemonitor.monitoring.coreos.com/odh-model-controller-metrics-monitor created [e2e-llm-inference-service] template.template.openshift.io/autogluon-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/guardrails-detector-huggingface-serving-template created [e2e-llm-inference-service] template.template.openshift.io/kserve-ovms created [e2e-llm-inference-service] template.template.openshift.io/mlserver-cuda-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/mlserver-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cpu-x86-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-cuda-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-gaudi-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-multinode-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-rocm-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-ppc64le-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-s390x-runtime-template-fast-2 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template-fast-1 created [e2e-llm-inference-service] template.template.openshift.io/vllm-spyre-x86-runtime-template-fast-2 created [e2e-llm-inference-service] mutatingwebhookconfiguration.admissionregistration.k8s.io/mutating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] validatingwebhookconfiguration.admissionregistration.k8s.io/validating.odh-model-controller.opendatahub.io created [e2e-llm-inference-service] Waiting for deployment "odh-model-controller" rollout to finish: 0 of 1 updated replicas are available... [e2e-llm-inference-service] deployment "odh-model-controller" successfully rolled out [e2e-llm-inference-service] networkpolicy.networking.k8s.io/allow-all created [e2e-llm-inference-service] KServe setup complete (namespace: kserve) [e2e-llm-inference-service] Add testing models to SeaweedFS S3 storage ... [e2e-llm-inference-service] Waiting for SeaweedFS deployment to be ready... [e2e-llm-inference-service] deployment "seaweedfs" successfully rolled out [e2e-llm-inference-service] S3 init job not completed, re-creating... [e2e-llm-inference-service] job.batch/s3-init replaced [e2e-llm-inference-service] Waiting for S3 init job to complete... [e2e-llm-inference-service] job.batch/s3-init condition met [e2e-llm-inference-service] Prepare CI namespace and install ServingRuntimes [e2e-llm-inference-service] Setting up CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Tearing down CI namespace: kserve-ci-e2e-test [e2e-llm-inference-service] Namespace kserve-ci-e2e-test does not exist, skipping deletion [e2e-llm-inference-service] CI namespace teardown complete [e2e-llm-inference-service] Creating namespace kserve-ci-e2e-test [e2e-llm-inference-service] namespace/kserve-ci-e2e-test created [e2e-llm-inference-service] Applying S3 artifact secret [e2e-llm-inference-service] secret/mlpipeline-s3-artifact created [e2e-llm-inference-service] Applying storage-config secret [e2e-llm-inference-service] secret/storage-config created [e2e-llm-inference-service] Applying SeaweedFS S3 credentials secret [e2e-llm-inference-service] secret/seaweedfs-s3-creds created [e2e-llm-inference-service] Linking seaweedfs-s3-creds to default service account [e2e-llm-inference-service] Creating odh-trusted-ca-bundle configmap [e2e-llm-inference-service] configmap/odh-trusted-ca-bundle created [e2e-llm-inference-service] Installing ServingRuntimes [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-autogluonserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-huggingfaceserver-multinode created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-lgbserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-mlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-paddleserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-pmmlserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-predictiveserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-sklearnserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tensorflow-serving created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-torchserve created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-tritonserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-vllmserver created [e2e-llm-inference-service] servingruntime.serving.kserve.io/kserve-xgbserver created [e2e-llm-inference-service] CI namespace setup complete [e2e-llm-inference-service] Setup complete [e2e-llm-inference-service] === E2E cluster / operator summary === [e2e-llm-inference-service] Client Version: 4.20.11 [e2e-llm-inference-service] Kustomize Version: v5.6.0 [e2e-llm-inference-service] Server Version: 4.21.25 [e2e-llm-inference-service] Kubernetes Version: v1.34.9 [e2e-llm-inference-service] ClusterVersion desired: 4.21.25 [e2e-llm-inference-service] ClusterVersion history (latest): 4.21.25 (Completed) [e2e-llm-inference-service] CSVs in kuadrant-system: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-keda: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] custom-metrics-autoscaler.v2.19.0-2 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in cert-manager-operator: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-lws-operator: [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] cert-manager-operator.v1.20.0 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] leader-worker-set.v1.0.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] CSVs in openshift-operators (ODH / shared operators, filtered): [e2e-llm-inference-service] authorino-operator.v1.4.1 Succeeded [e2e-llm-inference-service] dns-operator.v1.4.0 Succeeded [e2e-llm-inference-service] limitador-operator.v1.4.0 Succeeded [e2e-llm-inference-service] rhcl-operator.v1.4.1 Succeeded [e2e-llm-inference-service] Kuadrant / Authorino (diagnostics): [e2e-llm-inference-service] CRD kuadrants.kuadrant.io versions: v1beta1 served=true storage=true [e2e-llm-inference-service] Subscriptions in kuadrant-system: [e2e-llm-inference-service] authorino-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators authorino-operator.v1.4.1 [e2e-llm-inference-service] dns-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators dns-operator.v1.4.0 [e2e-llm-inference-service] limitador-operator-stable-redhat-operators-openshift-marketplace stable redhat-operators limitador-operator.v1.4.0 [e2e-llm-inference-service] rhcl-operator stable redhat-operators rhcl-operator.v1.4.1 [e2e-llm-inference-service] Kuadrant CR conditions (kuadrant/kuadrant-system): [e2e-llm-inference-service] Ready=True (Ready) [e2e-llm-inference-service] KServe deployments in kserve: [e2e-llm-inference-service] kserve-controller-manager: ready=1 image=quay.io/opendatahub/kserve-controller@sha256:3408a56bab73a176a3dc16d9a410a129dabf9c66a9bc9e7717bfc1996a904598 [e2e-llm-inference-service] imageID: quay.io/opendatahub/kserve-controller@sha256:1473cce55402f2847a0358f92b8ff08f9ed65a1b55d98f03cb01de6483034e41 [e2e-llm-inference-service] odh-model-controller: ready=1 image=quay.io/opendatahub/odh-model-controller:fast [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-model-controller@sha256:13d9671f9e4ff9e000e589178e1ad29625a50589ca69cbb353a478bfd4dc13cb [e2e-llm-inference-service] llmisvc-controller-manager: ready=1 image=quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0dc021ec33e3c2d04be2aaadffc003a21b60f42be583c38888b13b726ab7c5d0 [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:0dc021ec33e3c2d04be2aaadffc003a21b60f42be583c38888b13b726ab7c5d0 [e2e-llm-inference-service] === End E2E cluster / operator summary === [e2e-llm-inference-service] /workspace/source [e2e-llm-inference-service] CA certificate extracted [e2e-llm-inference-service] REQUESTS_CA_BUNDLE=/tmp/ca.crt [e2e-llm-inference-service] Run E2E tests: llminferenceservice and cluster_cpu and not autoscaling and not tracing [e2e-llm-inference-service] Starting E2E functional tests ... [e2e-llm-inference-service] Parallelism requested for pytest is 2 [e2e-llm-inference-service] ============================= test session starts ============================== [e2e-llm-inference-service] platform linux -- Python 3.11.13, pytest-7.4.4, pluggy-1.5.0 -- /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] cachedir: .pytest_cache [e2e-llm-inference-service] metadata: {'Python': '3.11.13', 'Platform': 'Linux-5.14.0-570.122.1.el9_6.x86_64-x86_64-with-glibc2.34', 'Packages': {'pytest': '7.4.4', 'pluggy': '1.5.0'}, 'Plugins': {'cov': '5.0.0', 'anyio': '4.9.0', 'asyncio': '0.23.8', 'httpx': '0.30.0', 'json-report': '1.5.0', 'xdist': '3.6.1', 'metadata': '3.1.1'}, 'PLATFORM': 'el9'} [e2e-llm-inference-service] rootdir: /workspace/source/test/e2e [e2e-llm-inference-service] configfile: pytest.ini [e2e-llm-inference-service] plugins: cov-5.0.0, anyio-4.9.0, asyncio-0.23.8, httpx-0.30.0, json-report-1.5.0, xdist-3.6.1, metadata-3.1.1 [e2e-llm-inference-service] asyncio: mode=Mode.STRICT [e2e-llm-inference-service] created: 2/2 workers [e2e-llm-inference-service] 2 workers [60 items] [e2e-llm-inference-service] [e2e-llm-inference-service] scheduling tests via WorkStealingScheduling [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_flow_control.py::test_flow_control_smoke[cluster_cpu-cluster_single_node-flow-control-utilization-detector] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] 2026-07-20 10:11:00.536 6422 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] 2026-07-20 10:11:00.536 6419 kserve INFO [conftest.py:configure_logger():40] Logger configured [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_flow_control.py::test_flow_control_smoke[cluster_cpu-cluster_single_node-flow-control-utilization-detector] [e2e-llm-inference-service] llmisvc/test_flow_control.py::test_flow_control_smoke[cluster_cpu-cluster_single_node-flow-control-concurrency-detector] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_flow_control.py::test_flow_control_smoke[cluster_cpu-cluster_single_node-flow-control-concurrency-detector] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] 2026-07-20 10:13:28.027 6419 kserve.trace Checking Gateway router-gateway-1 in namespace e2e-test-gateway-section-name-propagation-49b5edd9 [e2e-llm-inference-service] 2026-07-20 10:13:28.027 6419 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace e2e-test-gateway-section-name-propagation-49b5edd9 [e2e-llm-inference-service] 2026-07-20 10:13:28.052 6419 kserve.trace Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:13:28.052 6419 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:13:28.060 6419 kserve.trace ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:13:28.060 6419 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-with-section-name] [e2e-llm-inference-service] llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] 2026-07-20 10:13:37.491 6419 kserve.trace Checking Gateway router-gateway-1 in namespace e2e-test-gateway-section-name-propagation-94799d44 [e2e-llm-inference-service] 2026-07-20 10:13:37.491 6419 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace e2e-test-gateway-section-name-propagation-94799d44 [e2e-llm-inference-service] 2026-07-20 10:13:37.521 6419 kserve.trace Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:13:37.521 6419 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:13:37.529 6419 kserve.trace ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:13:37.529 6419 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_gateway_section_name.py::test_gateway_section_name_propagation[cluster_single_node-cluster_cpu-without-section-name] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_enabled_requires_token[cluster_cpu-cluster_single_node-auth-enabled-default] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] [gw0] FAILED llmisvc/test_llm_auth.py::test_llm_auth_enabled_requires_token[cluster_cpu-cluster_single_node-auth-enabled-default] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_invalid_token_rejected[cluster_cpu-cluster_single_node-auth-invalid-token] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_invalid_token_rejected[cluster_cpu-cluster_single_node-auth-invalid-token] [e2e-llm-inference-service] llmisvc/test_llm_auth.py::test_llm_auth_disabled_no_token_required[cluster_cpu-cluster_single_node-auth-disabled] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] [gw0] PASSED llmisvc/test_llm_auth.py::test_llm_auth_disabled_no_token_required[cluster_cpu-cluster_single_node-auth-disabled] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_hpa_deployment[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-hpa] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] [gw0] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_hpa_deployment[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-hpa] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_keda_deployment[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-keda] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_config_deletion.py::test_config_finalizer_added [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_config_deletion.py::test_config_finalizer_added [e2e-llm-inference-service] llmisvc/test_llm_inference_service_config_deletion.py::test_config_deletion_blocked_when_referenced [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_config_deletion.py::test_config_deletion_blocked_when_referenced [e2e-llm-inference-service] llmisvc/test_llm_inference_service_config_deletion.py::test_config_deletion_allowed_when_unreferenced [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_config_deletion.py::test_config_deletion_allowed_when_unreferenced [e2e-llm-inference-service] llmisvc/test_llm_inference_service_config_deletion.py::test_config_deletion_unblocked_after_service_deleted [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_config_deletion.py::test_config_deletion_unblocked_after_service_deleted [e2e-llm-inference-service] llmisvc/test_llm_inference_service_config_deletion.py::test_well_known_config_deletion_prevented_by_webhook [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_config_deletion.py::test_well_known_config_deletion_prevented_by_webhook [e2e-llm-inference-service] llmisvc/test_llm_inference_service_config_deletion.py::test_well_known_config_deletion_blocked_by_implicit_reference [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_config_deletion.py::test_well_known_config_deletion_blocked_by_implicit_reference [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha1_to_v1alpha2_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_v1alpha2_to_v1alpha1_conversion [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_criticality_preservation_via_annotations [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_lora_criticality_preservation [e2e-llm-inference-service] llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_conversion.py::TestLLMInferenceServiceConversion::test_round_trip_conversion_preserves_fields [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-single-lora-adapter-hf] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-single-lora-adapter-hf] [e2e-llm-inference-service] llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-multiple-lora-adapters] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_lora_adapters.py::test_llm_with_lora_adapters[cluster_cpu-multiple-lora-adapters] [e2e-llm-inference-service] llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_prestop_hook.py::test_prestop_hook[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_keda_deployment[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-keda] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_hpa_lws[cluster_cpu-cluster_multi_node-router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-hpa] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_prestop_hook.py::test_prestop_hook[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_rolling_upgrade.py::test_rolling_upgrade_coordination[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_rolling_upgrade.py::test_rolling_upgrade_coordination[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_storage_version_migration.py::TestStorageVersionMigration::test_storage_version_migration_after_simulated_upgrade [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] 2026-07-20 10:59:24.569 6422 kserve.trace Checking Gateway router-gateway-1 in namespace e2e-test-llm-inference-service-028f7809 [e2e-llm-inference-service] 2026-07-20 10:59:24.569 6422 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-1 in namespace e2e-test-llm-inference-service-028f7809 [e2e-llm-inference-service] 2026-07-20 10:59:24.602 6422 kserve.trace Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.602 6422 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.609 6422 kserve.trace ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.609 6422 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.609 6422 kserve.trace Checking HttpRoute router-route-1 in namespace e2e-test-llm-inference-service-028f7809 [e2e-llm-inference-service] 2026-07-20 10:59:24.609 6422 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-1 in namespace e2e-test-llm-inference-service-028f7809 [e2e-llm-inference-service] 2026-07-20 10:59:24.612 6422 kserve.trace Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.612 6422 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.628 6422 kserve.trace ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.628 6422 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-1 [e2e-llm-inference-service] 2026-07-20 10:59:24.628 6422 kserve.trace Checking HttpRoute router-route-2 in namespace e2e-test-llm-inference-service-028f7809 [e2e-llm-inference-service] 2026-07-20 10:59:24.628 6422 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-2 in namespace e2e-test-llm-inference-service-028f7809 [e2e-llm-inference-service] 2026-07-20 10:59:24.635 6422 kserve.trace Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-20 10:59:24.635 6422 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-20 10:59:24.648 6422 kserve.trace ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] 2026-07-20 10:59:24.648 6422 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-2 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw0] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_hpa_lws[cluster_cpu-cluster_multi_node-router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-hpa] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_keda_lws[cluster_cpu-cluster_multi_node-router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-keda] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] 2026-07-20 11:11:19.658 6422 kserve.trace Checking Gateway router-gateway-2 in namespace e2e-test-llm-inference-service-b5dd93e6 [e2e-llm-inference-service] 2026-07-20 11:11:19.658 6422 kserve.trace INFO [gw_api.py:create_or_update_gateway():34] Checking Gateway router-gateway-2 in namespace e2e-test-llm-inference-service-b5dd93e6 [e2e-llm-inference-service] 2026-07-20 11:11:19.688 6422 kserve.trace Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-20 11:11:19.688 6422 kserve.trace INFO [gw_api.py:create_or_update_gateway():62] Resource not found, creating Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-20 11:11:19.694 6422 kserve.trace ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-20 11:11:19.694 6422 kserve.trace INFO [gw_api.py:create_or_update_gateway():70] ✓ Successfully created Gateway router-gateway-2 [e2e-llm-inference-service] 2026-07-20 11:11:19.694 6422 kserve.trace Checking HttpRoute router-route-3 in namespace e2e-test-llm-inference-service-b5dd93e6 [e2e-llm-inference-service] 2026-07-20 11:11:19.694 6422 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-3 in namespace e2e-test-llm-inference-service-b5dd93e6 [e2e-llm-inference-service] 2026-07-20 11:11:19.698 6422 kserve.trace Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-20 11:11:19.698 6422 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-20 11:11:19.708 6422 kserve.trace ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-20 11:11:19.708 6422 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-3 [e2e-llm-inference-service] 2026-07-20 11:11:19.708 6422 kserve.trace Checking HttpRoute router-route-4 in namespace e2e-test-llm-inference-service-b5dd93e6 [e2e-llm-inference-service] 2026-07-20 11:11:19.708 6422 kserve.trace INFO [gw_api.py:create_or_update_route():121] Checking HttpRoute router-route-4 in namespace e2e-test-llm-inference-service-b5dd93e6 [e2e-llm-inference-service] 2026-07-20 11:11:19.714 6422 kserve.trace Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-20 11:11:19.714 6422 kserve.trace INFO [gw_api.py:create_or_update_route():149] Resource not found, creating HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-20 11:11:19.721 6422 kserve.trace ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] 2026-07-20 11:11:19.721 6422 kserve.trace INFO [gw_api.py:create_or_update_route():157] ✓ Successfully created HttpRoute router-route-4 [e2e-llm-inference-service] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] [gw0] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_keda_lws[cluster_cpu-cluster_multi_node-router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-keda] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] [gw1] PASSED llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] [e2e-llm-inference-service] =================================== FAILURES =================================== [e2e-llm-inference-service] __________ test_llm_auth_enabled_requires_token[auth-enabled-default] __________ [e2e-llm-inference-service] [gw0] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-single-cpu', 'model-fb-opt-125m'], prompt='KServe is a', service_name=... {'name': 'model-fb-opt-125m-auth-enabled-89f54b63'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] @pytest.mark.auth [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-single-cpu", [e2e-llm-inference-service] "model-fb-opt-125m", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="auth-enabled-test", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] ], [e2e-llm-inference-service] id="auth-enabled-default", [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_auth_enabled_requires_token(test_case: TestCase): # noqa: F811 [e2e-llm-inference-service] """ [e2e-llm-inference-service] Test that when auth is enabled (default): [e2e-llm-inference-service] - Requests WITH valid token succeed [e2e-llm-inference-service] - Requests WITHOUT token are rejected (401/403) [e2e-llm-inference-service] """ [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = KServeClient( [e2e-llm-inference-service] config_file=os.environ.get("KUBECONFIG", "~/.kube/config"), [e2e-llm-inference-service] client_configuration=client.Configuration(), [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] sa_name = f"{service_name}-test-sa" [e2e-llm-inference-service] test_failed = False [e2e-llm-inference-service] [e2e-llm-inference-service] # Enable auth for this test [e2e-llm-inference-service] if not test_case.llm_service.metadata.annotations: [e2e-llm-inference-service] test_case.llm_service.metadata.annotations = {} [e2e-llm-inference-service] test_case.llm_service.metadata.annotations[ [e2e-llm-inference-service] "security.opendatahub.io/enable-auth" [e2e-llm-inference-service] ] = "true" [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] # Create LLMInferenceService [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] # Create ServiceAccount with get+post access (required for inference-access and endpoint-access) [e2e-llm-inference-service] token = create_service_account_with_inference_access( [e2e-llm-inference-service] kserve_client, sa_name, service_name [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] service_url = get_llm_service_url(kserve_client, test_case.llm_service) [e2e-llm-inference-service] completion_url = f"{service_url}/v1/completions" [e2e-llm-inference-service] test_payload = { [e2e-llm-inference-service] "model": test_case.model_name, [e2e-llm-inference-service] "prompt": test_case.prompt, [e2e-llm-inference-service] "max_tokens": test_case.max_tokens, [e2e-llm-inference-service] } [e2e-llm-inference-service] [e2e-llm-inference-service] # Test 1: Request WITHOUT token should fail [e2e-llm-inference-service] logger.info("Testing request WITHOUT token (should fail)") [e2e-llm-inference-service] response_no_token = requests.post( [e2e-llm-inference-service] completion_url, [e2e-llm-inference-service] headers={"Content-Type": "application/json"}, [e2e-llm-inference-service] json=test_payload, [e2e-llm-inference-service] timeout=30, [e2e-llm-inference-service] ) [e2e-llm-inference-service] assert response_no_token.status_code in [ [e2e-llm-inference-service] 401, [e2e-llm-inference-service] 403, [e2e-llm-inference-service] ], ( [e2e-llm-inference-service] f"Expected 401/403 without token, got {response_no_token.status_code}: {response_no_token.text}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"✅ Request without token rejected: {response_no_token.status_code}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] # Test 2: Request WITH valid token should succeed. [e2e-llm-inference-service] # Retry to handle propagation delay — RBAC changes and Authorino auth policy [e2e-llm-inference-service] # caches may not reflect new bindings immediately after creation. [e2e-llm-inference-service] logger.info("Testing request WITH valid token (should succeed)") [e2e-llm-inference-service] token_headers = { [e2e-llm-inference-service] "Content-Type": "application/json", [e2e-llm-inference-service] "Authorization": f"Bearer {token}", [e2e-llm-inference-service] } [e2e-llm-inference-service] response_with_token = None [e2e-llm-inference-service] for attempt in range(24): # up to ~120s [e2e-llm-inference-service] response_with_token = requests.post( [e2e-llm-inference-service] completion_url, [e2e-llm-inference-service] headers=token_headers, [e2e-llm-inference-service] json=test_payload, [e2e-llm-inference-service] timeout=test_case.response_timeout, [e2e-llm-inference-service] ) [e2e-llm-inference-service] if response_with_token.status_code == 200: [e2e-llm-inference-service] break [e2e-llm-inference-service] if response_with_token.status_code in [401, 403]: [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"Attempt {attempt + 1}: got {response_with_token.status_code}, " [e2e-llm-inference-service] "waiting for RBAC propagation..." [e2e-llm-inference-service] ) [e2e-llm-inference-service] time.sleep(5) [e2e-llm-inference-service] else: [e2e-llm-inference-service] break [e2e-llm-inference-service] > assert response_with_token.status_code == 200, ( [e2e-llm-inference-service] f"Expected 200 with token, got {response_with_token.status_code}: {response_with_token.text}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Expected 200 with token, got 403: [e2e-llm-inference-service] E assert 403 == 200 [e2e-llm-inference-service] E + where 403 = .status_code [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_auth.py:336: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:74 Created test namespace e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret seaweedfs-s3-creds from kserve-ci-e2e-test to e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret storage-config from kserve-ci-e2e-test to e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:170 Patched default SA in e2e-test-llm-auth-enabled-requires-token-d4ed2c08 with secret seaweedfs-s3-creds [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:154 ConfigMap odh-kserve-custom-ca-bundle already exists in e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:151 Copied ConfigMap odh-trusted-ca-bundle from kserve-ci-e2e-test to e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig router-managed-auth-enabled-tes-210a8f79 in namespace e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig router-managed-auth-enabled-tes-210a8f79 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig router-managed-auth-enabled-tes-210a8f79 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig workload-single-cpu-auth-enable-a18fd8e2 in namespace e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig workload-single-cpu-auth-enable-a18fd8e2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig workload-single-cpu-auth-enable-a18fd8e2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig model-fb-opt-125m-auth-enabled-89f54b63 in namespace e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig model-fb-opt-125m-auth-enabled-89f54b63 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig model-fb-opt-125m-auth-enabled-89f54b63 [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_auth_enabled_requires_token] [2026-07-20T10:13:58.619817] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'workload-single-cpu', 'model-fb-opt-125m'], prompt='KServe is a', service_name='auth-enabled-test', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, namespace='e2e-test-llm-auth-enabled-requires-token-d4ed2c08', before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'auth-enabled-test', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-auth-enabled-requires-token-d4ed2c08', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-auth-enabled-tes-210a8f79'}, [e2e-llm-inference-service] {'name': 'workload-single-cpu-auth-enable-a18fd8e2'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-auth-enabled-89f54b63'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-07-20T10:13:58.633046] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'true'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'auth-enabled-test', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-auth-enabled-requires-token-d4ed2c08', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-auth-enabled-tes-210a8f79'}, [e2e-llm-inference-service] {'name': 'workload-single-cpu-auth-enable-a18fd8e2'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-auth-enabled-89f54b63'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-07-20T10:13:58.810335] end - ✅ in 0.177s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-07-20T10:13:58.810489] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'true'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'auth-enabled-test', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-auth-enabled-requires-token-d4ed2c08', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-auth-enabled-tes-210a8f79'}, [e2e-llm-inference-service] {'name': 'workload-single-cpu-auth-enable-a18fd8e2'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-auth-enabled-89f54b63'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'The following HTTPRoutes are not ready: [e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'severity': 'Info', 'status': 'False', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Inference Pool e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-inference-pool exists but no Gateway controller has accepted it yet', 'reason': 'WaitingForGateway', 'severity': 'Info', 'status': 'False', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'The following HTTPRoutes are not ready: [e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'The following HTTPRoutes are not ready: [e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-route: "False" (reason "InvalidKind", message "referencing unsupported backendRef: group \\"inference.networking.x-k8s.io\\" kind \\"InferencePool\\"")]', 'reason': 'HTTPRoutesNotReady', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Deployment rollout in progress', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:14:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-07-20T10:14:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:14:18Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:14:18Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:14:18Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:14:18Z', 'severity': 'Info', 'status': 'True', 'type': 'HTTPRoutesReady'}, {'lastTransitionTime': '2026-07-20T10:14:18Z', 'severity': 'Info', 'status': 'True', 'type': 'InferencePoolReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:14:18Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:14:41Z', 'status': 'True', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:14:41Z', 'severity': 'Info', 'status': 'True', 'type': 'SchedulerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:14:05Z', 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 1: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [wait_for_llm_isvc_ready] [2026-07-20T10:16:34.900138] end - ✅ in 156.089s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:95 ✅ Created ServiceAccount auth-enabled-test-test-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:117 ✅ Created Role auth-enabled-test-test-sa-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:146 ✅ Created RoleBinding auth-enabled-test-test-sa-binding [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:64 Using token audiences from AuthPolicy: ['https://prod-eaas-bucket.s3.us-east-1.amazonaws.com/7e8aaa8ec2d9'] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:182 ✅ Created token for ServiceAccount auth-enabled-test-test-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [get_llm_service_url] [2026-07-20T10:16:34.998237] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': {'security.opendatahub.io/enable-auth': 'true'}, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'auth-enabled-test', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-auth-enabled-requires-token-d4ed2c08', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-auth-enabled-tes-210a8f79'}, [e2e-llm-inference-service] {'name': 'workload-single-cpu-auth-enable-a18fd8e2'}, [e2e-llm-inference-service] {'name': 'model-fb-opt-125m-auth-enabled-89f54b63'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [get_llm_service_url] [2026-07-20T10:16:35.008430] end - ✅ in 0.010s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:293 Testing request WITHOUT token (should fail) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:306 ✅ Request without token rejected: 401 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:313 Testing request WITH valid token (should succeed) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 1: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 2: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 3: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 4: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 5: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 6: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 7: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 8: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 9: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 10: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 11: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 12: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 13: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 14: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 15: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 16: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 17: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 18: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 19: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 20: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 21: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 22: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 23: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:329 Attempt 24: got 403, waiting for RBAC propagation... [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_auth.py:345 ❌ ERROR: Failed test for auth-enabled-test: Expected 200 with token, got 403: [e2e-llm-inference-service] assert 403 == 200 [e2e-llm-inference-service] + where 403 = .status_code [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:243 # Diagnostics for 'auth-enabled-test' in 'e2e-test-llm-auth-enabled-requires-token-d4ed2c08' [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:244 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:247 # LLMInferenceService auth-enabled-test [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:256 apiVersion: serving.kserve.io/v1alpha1 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] security.opendatahub.io/enable-auth: 'true' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:13:58Z' [e2e-llm-inference-service] finalizers: [e2e-llm-inference-service] - serving.kserve.io/llmisvc-finalizer [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:security.opendatahub.io/enable-auth: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:baseRefs: {} [e2e-llm-inference-service] manager: OpenAPI-Generator [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:13:58Z' [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:finalizers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] v:"serving.kserve.io/llmisvc-finalizer": {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:13:59Z' [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:addresses: {} [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-decode-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-decode-worker-data-parallel: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-prefill-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-prefill-worker-data-parallel: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-router-route: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-scheduler: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-scheduler-latency-predictor: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-template: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-tracing: {} [e2e-llm-inference-service] f:serving.kserve.io/config-llm-worker-data-parallel: {} [e2e-llm-inference-service] f:appliedConfigs: {} [e2e-llm-inference-service] f:conditions: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:router: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:gateways: {} [e2e-llm-inference-service] f:scheduler: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:inferencePool: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:service: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:url: {} [e2e-llm-inference-service] f:workloads: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:primary: {} [e2e-llm-inference-service] f:scheduler: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-20T10:16:34Z' [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] resourceVersion: '28795' [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] baseRefs: [e2e-llm-inference-service] - name: router-managed-auth-enabled-tes-210a8f79 [e2e-llm-inference-service] - name: workload-single-cpu-auth-enable-a18fd8e2 [e2e-llm-inference-service] - name: model-fb-opt-125m-auth-enabled-89f54b63 [e2e-llm-inference-service] model: [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uri: '' [e2e-llm-inference-service] status: [e2e-llm-inference-service] addresses: [e2e-llm-inference-service] - name: gateway-external-model-routing [e2e-llm-inference-service] url: http://aa8e6e695d7614e028ae8fd0983dbce3-1759906753.us-east-1.elb.amazonaws.com/ [e2e-llm-inference-service] - name: gateway-external [e2e-llm-inference-service] url: http://aa8e6e695d7614e028ae8fd0983dbce3-1759906753.us-east-1.elb.amazonaws.com/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test [e2e-llm-inference-service] - name: gateway-external [e2e-llm-inference-service] url: http://aa8e6e695d7614e028ae8fd0983dbce3-1759906753.us-east-1.elb.amazonaws.com/publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] - name: gateway-internal-model-routing [e2e-llm-inference-service] url: http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/ [e2e-llm-inference-service] - name: gateway-internal [e2e-llm-inference-service] url: http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test [e2e-llm-inference-service] - name: gateway-internal [e2e-llm-inference-service] url: http://openshift-ai-inference-openshift-default.openshift-ingress.svc.cluster.local/publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/config-llm-decode-template: kserve-config-llm-decode-template [e2e-llm-inference-service] serving.kserve.io/config-llm-decode-worker-data-parallel: kserve-config-llm-decode-worker-data-parallel [e2e-llm-inference-service] serving.kserve.io/config-llm-prefill-template: kserve-config-llm-prefill-template [e2e-llm-inference-service] serving.kserve.io/config-llm-prefill-worker-data-parallel: kserve-config-llm-prefill-worker-data-parallel [e2e-llm-inference-service] serving.kserve.io/config-llm-router-route: kserve-config-llm-router-route [e2e-llm-inference-service] serving.kserve.io/config-llm-scheduler: kserve-config-llm-scheduler [e2e-llm-inference-service] serving.kserve.io/config-llm-scheduler-latency-predictor: kserve-config-llm-scheduler-latency-predictor [e2e-llm-inference-service] serving.kserve.io/config-llm-template: kserve-config-llm-template [e2e-llm-inference-service] serving.kserve.io/config-llm-tracing: kserve-config-llm-tracing [e2e-llm-inference-service] serving.kserve.io/config-llm-worker-data-parallel: kserve-config-llm-worker-data-parallel [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:18Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: HTTPRoutesReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:18Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: InferencePoolReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:16:34Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: MainWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:05Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: PresetsCombined [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:16:34Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Ready [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:41Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: RouterReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:41Z' [e2e-llm-inference-service] severity: Info [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: SchedulerWorkloadReady [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:16:34Z' [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: WorkloadsReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] url: http://aa8e6e695d7614e028ae8fd0983dbce3-1759906753.us-east-1.elb.amazonaws.com/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:68 TIME NAMESPACE SOURCE TYPE REASON MESSAGE [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:69 -------------------------------------------------------------------------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 None e2e-test-llm-auth-enabled-requires-token-d4ed2c08 Normal Scheduled Successfully assigned e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-6c997987dd-r59r7 to ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 multus Normal AddedInterface Add eth0 [10.134.0.34/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:07 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Pulling Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:38 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Pulled Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 31.079s (31.079s including waiting). Image size: 3531177328 bytes. [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:38 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:38 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:16:23 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Warning Unhealthy Startup probe failed: Get "https://10.134.0.34:8000/health": dial tcp 10.134.0.34:8000: connect: connection refused [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 replicaset-controller Normal SuccessfulCreate Created pod: auth-enabled-test-kserve-6c997987dd-r59r7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 None e2e-test-llm-auth-enabled-requires-token-d4ed2c08 Normal Scheduled Successfully assigned e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-794498b458-mt7dc to ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:04 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 multus Normal AddedInterface Add eth0 [10.134.0.35/23] from ovn-kubernetes [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:04 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Pulled Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:04 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Created Created container: storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:04 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Started Started container storage-initializer [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:05 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Pulled Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:05 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Created Created container: main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:05 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 kubelet/ip-10-0-138-196.ec2.internal Normal Started Started container main [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 replicaset-controller Normal SuccessfulCreate Created pod: auth-enabled-test-kserve-router-scheduler-794498b458-mt7dc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-enabled-test-kserve-router-scheduler-794498b458 from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 deployment-controller Normal ScalingReplicaSet Scaled up replica set auth-enabled-test-kserve-6c997987dd from 0 to 1 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.Secret e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.Deployment e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.Service e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.ServiceAccount e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-epp-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.ClusterRoleBinding /e6eba5ecf557d59a5eb263733b3690452-auth-enabled-test-epp-auth-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.Role e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-epp-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.RoleBinding e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-epp-rb [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.Deployment e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:03 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created Created v1.Service e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-epp-service [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:17 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Created (combined from similar events): Created v1.DestinationRule e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-shadow-svc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:17 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Updated Updated v1.Secret e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:14:17 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal Updated Updated v1.HTTPRoute e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-route [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:80 2026-07-20 10:16:34 e2e-test-llm-auth-enabled-requires-token-d4ed2c08 LLMInferenceServiceController Normal LLMInferenceServiceReady LLMInferenceService [auth-enabled-test] is Ready [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:172 ### Pod auth-enabled-test-kserve-6c997987dd-r59r7 (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:212 #### init-container 'storage-initializer' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:225 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:226 2026-07-20 10:14:03.944 1 storage.initializer INFO [initializer-entrypoint:():17] Initializing, args: (src_uri, dest_path): [('hf://facebook/opt-125m', '/mnt/models')] [e2e-llm-inference-service] 2026-07-20 10:14:03.944 1 storage.initializer INFO [kserve_storage.py:download():168] Copying contents of hf://facebook/opt-125m to local [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/wPaCkH-WbT7GsmxMKKrNZTV4nSM=.ac481c8eb05e4d2496fbe076a38a7b4835dd733d.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_28242213-bacc-41c9-9638-d440e85a47c2'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/5HHJ6px3_ZRDOG3OxNZMhuycwOk=.a591333512516f58bf2002045dece909a0ccdb8b.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_41d16aa2-9f16-43ff-9779-46bc3d165ab7'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/Xn7B-BWUGOee2Y6hCZtEhtFu4BE=.38c05904caf6e5b9f04ecda5c973d77e6c1da151.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_a4cdb02f-9fbd-40ff-97ca-cff0d17aebf6'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/8_PA_wEVGiVa2goH2H4KQOQpvVY=.b3fb716a3024261980becb2382e31a3780985130.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_b26b40ec-b18f-4be3-bc51-d99d9cb0bb6c'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/gPcsVCQDYDHk-_n0G9uADl7PXIM=.61c60ec52ed43038fff0fbbd68b080c94b0d94b4c8458dbd65965f9b17631c89.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_d30b9608-3855-4470-b501-5c22743c9c33'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/3EVKVggOldJcKSsGjSdoUCN1AyQ=.cf739e3ba86db7791ebab2828cc34b8a5acd3a86.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_c2ba8d82-08b6-4c3a-940a-bddcb154b30b'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/PtHk0z_I45atnj23IIRhTExwT3w=.226b0752cac7789c48f0cb3ec53eda48b7be36cc.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_16865ce9-41e8-4bf4-b24e-f050c2abb891'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/Q1p2l2BzM1m6P5jKvr8WTq1TUio=.2d74da6615135c58cf3cf9ad4cb11e7c613ff9e55fe658a47ab83b6c8d1174a9.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_02e852e4-ba69-433d-a84e-f25d3c6480b4'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/ahkChHUJFxEmOdq5GDFEmerRzCY=.5dfa36546b8eddce0e04df3133c30df43fcc3828.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_e8db9a38-8fae-4b62-8123-d0e59bc077aa'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/a7eHxRFT3OeMBIFg52k2nfj5m7w=.db7090b0c8b34dd957a7e0656c718f978f9203cc874018f37dda44108be5970a.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_c5790a3d-0c4e-49e2-902e-2e9701df7775'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/vzaExXFZNBay89bvlQv-ZcI6BTg=.27c24ca9d908d0b678b20c698aeb9e950c44d865.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_ab476732-ccfd-48c7-bfad-91f4c2e8beea'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/j3m-Hy6QvBddw8RXA1uSWl1AJ0c=.0a39732b2d8be8e493cab3da68b68cc3e28221de.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_1ad0a9ee-6c82-4645-993a-5bf231723371'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] 2026-07-20 10:14:07.156 1 storage.initializer INFO [kserve_storage.py:download():243] Successfully copied hf://facebook/opt-125m to /mnt/models [e2e-llm-inference-service] 2026-07-20 10:14:07.156 1 storage.initializer INFO [kserve_storage.py:download():244] Model downloaded in 3.2124042840000584 seconds. [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:212 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:225 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:226 (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] Failed to load AutoTokenizer chat template for /mnt/models [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] Traceback (most recent call last): [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] File "/opt/venv/lib/python3.12/site-packages/vllm/renderers/hf.py", line 120, in resolve_chat_template [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] return tokenizer.get_chat_template(chat_template, tools=tools) [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] File "/opt/venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1825, in get_chat_template [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] raise ValueError( [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:141] There is no chat template fallback for /mnt/models [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [renderers/hf.py:314] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] Failed to load AutoTokenizer chat template for /mnt/models [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] Traceback (most recent call last): [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] File "/opt/venv/lib/python3.12/site-packages/vllm/renderers/hf.py", line 120, in resolve_chat_template [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] return tokenizer.get_chat_template(chat_template, tools=tools) [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] File "/opt/venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1825, in get_chat_template [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] raise ValueError( [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/hf.py:122] ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:29 [renderers/base.py:205] This model does not support chat template. [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/openai/api_server.py:594] Starting vLLM server on https://0.0.0.0:8000 [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:37] Available routes are: [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /openapi.json, Methods: HEAD, GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /docs, Methods: HEAD, GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /docs/oauth2-redirect, Methods: HEAD, GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /redoc, Methods: HEAD, GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /tokenize, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /detokenize, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /load, Methods: GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /version, Methods: GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /health, Methods: GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /metrics, Methods: GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/models, Methods: GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /ping, Methods: GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /ping, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /invocations, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/chat/completions, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/chat/completions/batch, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/responses, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/completions, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/messages, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/messages/count_tokens, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /inference/v1/generate, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /scale_elastic_ep, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/chat/completions/render, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/launcher.py:46] Route: /v1/completions/render, Methods: POST [e2e-llm-inference-service] (APIServer pid=1) INFO: Started server process [1] [e2e-llm-inference-service] (APIServer pid=1) INFO: Waiting for application startup. [e2e-llm-inference-service] (APIServer pid=1) INFO 07-20 10:16:29 [entrypoints/ssl.py:60] SSLCertRefresher monitors files: ['/var/run/kserve/tls/tls.key', '/var/run/kserve/tls/tls.crt'] [e2e-llm-inference-service] (APIServer pid=1) INFO: Application startup complete. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:33 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:33 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:34 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:35 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:36 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:37 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:38 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:39 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:39 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:40 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:41 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:42 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:43 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:43 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:44 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:45 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:46 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:47 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:48 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:49 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:49 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:50 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:51 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:52 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:53 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:53 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:54 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:55 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:56 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:57 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:58 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:59 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:16:59 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:00 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:01 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:02 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:03 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:03 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:04 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:05 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:06 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:07 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:08 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:09 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:09 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:10 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:11 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:12 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:13 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:13 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:14 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:15 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:16 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:17 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:18 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:19 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:19 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:20 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:21 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:22 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:23 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:23 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:24 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:25 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:26 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:27 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:28 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:29 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:29 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:30 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:31 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:32 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:33 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:33 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:34 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:35 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:36 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:37 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:38 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:39 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:39 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:40 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:41 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:42 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:43 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:43 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:44 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:45 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:46 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:47 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:48 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:49 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:49 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:50 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:51 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:52 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:53 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:53 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:54 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:55 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:56 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:57 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:58 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:59 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:17:59 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:00 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:01 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:02 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:03 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:03 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:04 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:05 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:06 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:07 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:08 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:09 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:09 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:10 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:11 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:12 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:13 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:13 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:14 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:15 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:16 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:17 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:18 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:19 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:19 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:20 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:21 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:22 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:23 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:23 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:24 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:25 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:26 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:27 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:28 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:29 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:29 [v1/metrics/loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:30 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:31 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:32 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:33 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:33 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:34 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] (APIServer pid=1) DEBUG 07-20 10:18:35 [v1/engine/async_llm.py:875] Called check_health. [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:172 ### Pod auth-enabled-test-kserve-router-scheduler-794498b458-mt7dc (phase=Running) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:212 #### init-container 'storage-initializer' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:225 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:226 2026-07-20 10:14:04.411 1 storage.initializer INFO [initializer-entrypoint:():17] Initializing, args: (src_uri, dest_path): [('hf://facebook/opt-125m', '/mnt/models')] [e2e-llm-inference-service] 2026-07-20 10:14:04.411 1 storage.initializer INFO [kserve_storage.py:download():168] Copying contents of hf://facebook/opt-125m to local [e2e-llm-inference-service] 2026-07-20 10:14:04.411 1 storage.initializer INFO [kserve_storage.py:download():171] Allow patterns: ['tokenizer.json', 'tokenizer_config.json', 'special_tokens_map.json', 'vocab.json', 'merges.txt', 'config.json', 'generation_config.json'] [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/8_PA_wEVGiVa2goH2H4KQOQpvVY=.b3fb716a3024261980becb2382e31a3780985130.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_05175d88-627b-42d7-95f5-15d2a0cd3e61'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/3EVKVggOldJcKSsGjSdoUCN1AyQ=.cf739e3ba86db7791ebab2828cc34b8a5acd3a86.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_139f83f9-de2c-427a-b065-c6ecd9a0edcc'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/PtHk0z_I45atnj23IIRhTExwT3w=.226b0752cac7789c48f0cb3ec53eda48b7be36cc.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_b108cfa0-dd57-4fce-8585-b4a3e260e167'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/ahkChHUJFxEmOdq5GDFEmerRzCY=.5dfa36546b8eddce0e04df3133c30df43fcc3828.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_585d98cf-6c1d-4de4-a8e0-d6cc3192fa24'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/vzaExXFZNBay89bvlQv-ZcI6BTg=.27c24ca9d908d0b678b20c698aeb9e950c44d865.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_0451419e-2d24-449d-babf-96b97efa902a'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] Could not set the permissions on the file '/mnt/models/.cache/huggingface/download/j3m-Hy6QvBddw8RXA1uSWl1AJ0c=.0a39732b2d8be8e493cab3da68b68cc3e28221de.incomplete'. Error: [Errno 13] Permission denied: '/mnt/tmp_77725a73-3d73-4461-a276-258ee9ba6718'. [e2e-llm-inference-service] Continuing without setting permissions. [e2e-llm-inference-service] 2026-07-20 10:14:04.844 1 storage.initializer INFO [kserve_storage.py:download():243] Successfully copied hf://facebook/opt-125m to /mnt/models [e2e-llm-inference-service] 2026-07-20 10:14:04.844 1 storage.initializer INFO [kserve_storage.py:download():244] Model downloaded in 0.43331839799998306 seconds. [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:212 #### container 'main' (restarts=0) [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:225 # -- logs (current) -- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:226 {"level":"info","ts":1784542625.339014,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"7d9f0b46-729b-45c7-bb9d-f32b94c9b83a","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542630.3789175,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"1d783fe7-99d0-44fa-abd4-b23fc15872e1"} [e2e-llm-inference-service] {"level":"info","ts":1784542630.379191,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"1d783fe7-99d0-44fa-abd4-b23fc15872e1","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542635.4110239,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"11920ad8-0a28-4668-bba7-c777f4596279"} [e2e-llm-inference-service] {"level":"info","ts":1784542635.4112878,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"11920ad8-0a28-4668-bba7-c777f4596279","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542640.4493217,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"1e943d96-2cc1-4ecd-9105-240c4beb0087"} [e2e-llm-inference-service] {"level":"info","ts":1784542640.4496136,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"1e943d96-2cc1-4ecd-9105-240c4beb0087","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542645.4903967,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"c1235f0b-f444-4e6f-b09d-7b4d63e87e42"} [e2e-llm-inference-service] {"level":"info","ts":1784542645.490796,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"c1235f0b-f444-4e6f-b09d-7b4d63e87e42","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542650.5274007,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"60dc4bd1-ca41-9ff7-b5fa-b506d7cb24dd"} [e2e-llm-inference-service] {"level":"info","ts":1784542650.5276785,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"60dc4bd1-ca41-9ff7-b5fa-b506d7cb24dd","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542655.5626302,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"9cafd352-6381-4154-a0f5-d95896b01a1e"} [e2e-llm-inference-service] {"level":"info","ts":1784542655.5628743,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"9cafd352-6381-4154-a0f5-d95896b01a1e","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542660.6008673,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"0ca95129-46cd-4f66-b71b-56a96298b36b"} [e2e-llm-inference-service] {"level":"info","ts":1784542660.6011362,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"0ca95129-46cd-4f66-b71b-56a96298b36b","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542665.638852,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"50b118c0-dc4b-4266-a835-a459f2ab0f29"} [e2e-llm-inference-service] {"level":"info","ts":1784542665.639124,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"50b118c0-dc4b-4266-a835-a459f2ab0f29","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542670.6800718,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"e4ce6930-3030-41e3-85b7-7f3651e0fb94"} [e2e-llm-inference-service] {"level":"info","ts":1784542670.6803453,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"e4ce6930-3030-41e3-85b7-7f3651e0fb94","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542675.7163806,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"0cb5a3d4-6bba-42ce-98fd-e0cffdb317bb"} [e2e-llm-inference-service] {"level":"info","ts":1784542675.7167394,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"0cb5a3d4-6bba-42ce-98fd-e0cffdb317bb","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542680.7514887,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"74be1cd2-efad-48b5-a5a3-43abed26224c"} [e2e-llm-inference-service] {"level":"info","ts":1784542680.751778,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"74be1cd2-efad-48b5-a5a3-43abed26224c","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542685.79498,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"df499c9c-2eb9-40df-9fe2-73ac6923437e"} [e2e-llm-inference-service] {"level":"info","ts":1784542685.7952704,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"df499c9c-2eb9-40df-9fe2-73ac6923437e","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542690.8359518,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"df825f7d-6021-443c-b238-cff622e823a6"} [e2e-llm-inference-service] {"level":"info","ts":1784542690.8362324,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"df825f7d-6021-443c-b238-cff622e823a6","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] { [e2e-llm-inference-service] "Name": "gateway.request_orchestration", [e2e-llm-inference-service] "SpanContext": { [e2e-llm-inference-service] "TraceID": "80796c668c448f3f023473f5e1780a3a", [e2e-llm-inference-service] "SpanID": "74b46c52a80d210f", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "Parent": { [e2e-llm-inference-service] "TraceID": "80796c668c448f3f023473f5e1780a3a", [e2e-llm-inference-service] "SpanID": "5934298508e2be30", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "SpanKind": 2, [e2e-llm-inference-service] "StartTime": "2026-07-20T10:18:10.8360579Z", [e2e-llm-inference-service] "EndTime": "2026-07-20T10:18:10.836174735Z", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "target_model", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "facebook/opt-125m" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "request_prio", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "INT64", [e2e-llm-inference-service] "Value": 0 [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "Events": null, [e2e-llm-inference-service] "Links": null, [e2e-llm-inference-service] "Status": { [e2e-llm-inference-service] "Code": "Unset", [e2e-llm-inference-service] "Description": "" [e2e-llm-inference-service] }, [e2e-llm-inference-service] "DroppedAttributes": 0, [e2e-llm-inference-service] "DroppedEvents": 0, [e2e-llm-inference-service] "DroppedLinks": 0, [e2e-llm-inference-service] "ChildSpanCount": 0, [e2e-llm-inference-service] "Resource": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.name", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "llm-d-epp" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.version", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "InstrumentationScope": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/requestcontrol", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] }, [e2e-llm-inference-service] "InstrumentationLibrary": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/requestcontrol", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] { [e2e-llm-inference-service] "Name": "gateway.request", [e2e-llm-inference-service] "SpanContext": { [e2e-llm-inference-service] "TraceID": "80796c668c448f3f023473f5e1780a3a", [e2e-llm-inference-service] "SpanID": "5934298508e2be30", [e2e-llm-inference-service] "TraceFlags": "01", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "Parent": { [e2e-llm-inference-service] "TraceID": "00000000000000000000000000000000", [e2e-llm-inference-service] "SpanID": "0000000000000000", [e2e-llm-inference-service] "TraceFlags": "00", [e2e-llm-inference-service] "TraceState": "", [e2e-llm-inference-service] "Remote": false [e2e-llm-inference-service] }, [e2e-llm-inference-service] "SpanKind": 2, [e2e-llm-inference-service] "StartTime": "2026-07-20T10:18:10.835999717Z", [e2e-llm-inference-service] "EndTime": "2026-07-20T10:18:10.850505963Z", [e2e-llm-inference-service] "Attributes": null, [e2e-llm-inference-service] "Events": null, [e2e-llm-inference-service] "Links": null, [e2e-llm-inference-service] "Status": { [e2e-llm-inference-service] "Code": "Unset", [e2e-llm-inference-service] "Description": "" [e2e-llm-inference-service] }, [e2e-llm-inference-service] "DroppedAttributes": 0, [e2e-llm-inference-service] "DroppedEvents": 0, [e2e-llm-inference-service] "DroppedLinks": 0, [e2e-llm-inference-service] "ChildSpanCount": 1, [e2e-llm-inference-service] "Resource": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.name", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "llm-d-epp" [e2e-llm-inference-service] } [e2e-llm-inference-service] }, [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "service.version", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ], [e2e-llm-inference-service] "InstrumentationScope": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/handlers", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] }, [e2e-llm-inference-service] "InstrumentationLibrary": { [e2e-llm-inference-service] "Name": "llm-d-router/pkg/epp/handlers", [e2e-llm-inference-service] "Version": "", [e2e-llm-inference-service] "SchemaURL": "", [e2e-llm-inference-service] "Attributes": [ [e2e-llm-inference-service] { [e2e-llm-inference-service] "Key": "commit-sha", [e2e-llm-inference-service] "Value": { [e2e-llm-inference-service] "Type": "STRING", [e2e-llm-inference-service] "Value": "unknown" [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] ] [e2e-llm-inference-service] } [e2e-llm-inference-service] } [e2e-llm-inference-service] {"level":"info","ts":1784542695.8722212,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"3030fdd5-d376-434e-9952-a5fa85cc373b"} [e2e-llm-inference-service] {"level":"info","ts":1784542695.87249,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"3030fdd5-d376-434e-9952-a5fa85cc373b","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542700.9098265,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"703e1792-6e32-4b68-b6a0-b6689abb9d11"} [e2e-llm-inference-service] {"level":"info","ts":1784542700.910097,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"703e1792-6e32-4b68-b6a0-b6689abb9d11","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542705.948907,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"9dac98ea-59b8-403c-88da-42fc38ad9d0e"} [e2e-llm-inference-service] {"level":"info","ts":1784542705.9491746,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"9dac98ea-59b8-403c-88da-42fc38ad9d0e","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] {"level":"info","ts":1784542710.9829562,"caller":"handlers/server.go:395","msg":"EPP received request","x-request-id":"93de1f4d-04e9-4ee8-b212-560c2d1bf6dd"} [e2e-llm-inference-service] {"level":"info","ts":1784542710.9832313,"caller":"handlers/server.go:656","msg":"EPP sent request body response(s) to proxy","x-request-id":"93de1f4d-04e9-4ee8-b212-560c2d1bf6dd","modelName":"facebook/opt-125m","targetModelName":"facebook/opt-125m"} [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-service [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: d5ced1b8-0c5c-442a-973f-2966c2598032 [e2e-llm-inference-service] resourceVersion: '27159' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpoints.kubernetes.io/managed-by: endpoint-controller [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:subsets: {} [e2e-llm-inference-service] subsets: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - ip: 10.134.0.35 [e2e-llm-inference-service] nodeName: ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler-794498b458-mt7dc [e2e-llm-inference-service] uid: b99ce9e9-0067-46b0-a74e-9b05501a3c02 [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Endpoints [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: b97cc1a9-ddc9-4cda-871b-5c505aabc1d1 [e2e-llm-inference-service] resourceVersion: '28783' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpoints.kubernetes.io/managed-by: endpoint-controller [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:subsets: {} [e2e-llm-inference-service] subsets: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - ip: 10.134.0.34 [e2e-llm-inference-service] nodeName: ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] name: auth-enabled-test-kserve-6c997987dd-r59r7 [e2e-llm-inference-service] uid: 55e3988c-bb7d-4c82-bb5d-f8bb06dcd862 [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Endpoints [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-6c997987dd-r59r7 [e2e-llm-inference-service] generateName: auth-enabled-test-kserve-6c997987dd- [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 55e3988c-bb7d-4c82-bb5d-f8bb06dcd862 [e2e-llm-inference-service] resourceVersion: '28779' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] pod-template-hash: 6c997987dd [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.134.0.34/23"],"mac_address":"0a:58:0a:86:00:22","gateway_ips":["10.134.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.134.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.134.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.134.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.134.0.1"}],"ip_address":"10.134.0.34/23","gateway_ip":"10.134.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.134.0.34\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:86:00:22\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: auth-enabled-test-kserve-6c997987dd [e2e-llm-inference-service] uid: 17474aac-8c70-466d-bfc0-ffdde13897f3 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-138-196 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"17474aac-8c70-466d-bfc0-ffdde13897f3"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"TORCHINDUCTOR_CACHE_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"USER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_CPU_KVCACHE_SPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_ENABLE_V1_MULTIPROCESSING"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"storage-initializer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"AWS_ACCESS_KEY_ID"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE_CONFIGMAP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_ENDPOINT_URL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_SECRET_ACCESS_KEY"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_ENABLE_HF_TRANSFER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_HIGH_PERFORMANCE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_NUM_CONCURRENT_RANGE_GETS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_ENDPOINT"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_USE_HTTPS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_VERIFY_SSL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"TOKIO_WORKER_THREADS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"kserve-provision-location"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:initContainerStatuses: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.134.0.34"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: kube-api-access-hcmjt [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] args: [e2e-llm-inference-service] - hf://facebook/opt-125m [e2e-llm-inference-service] - /mnt/models [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: TOKIO_WORKER_THREADS [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] - name: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] - name: S3_USE_HTTPS [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: S3_ENDPOINT [e2e-llm-inference-service] value: s3-service.kserve:8333 [e2e-llm-inference-service] - name: AWS_ENDPOINT_URL [e2e-llm-inference-service] value: http://s3-service.kserve:8333 [e2e-llm-inference-service] - name: S3_VERIFY_SSL [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: AWS_CA_BUNDLE [e2e-llm-inference-service] value: /etc/ssl/custom-certs/cabundle.crt [e2e-llm-inference-service] - name: AWS_CA_BUNDLE_CONFIGMAP [e2e-llm-inference-service] value: odh-kserve-custom-ca-bundle [e2e-llm-inference-service] - name: HF_HUB_ENABLE_HF_TRANSFER [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_HIGH_PERFORMANCE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_NUM_CONCURRENT_RANGE_GETS [e2e-llm-inference-service] value: '8' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] - name: kube-api-access-hcmjt [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000780000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/bash [e2e-llm-inference-service] - -c [e2e-llm-inference-service] - "if [ -f /etc/profile.d/ibm-aiu-setup.sh ]; then\n source /etc/profile.d/ibm-aiu-setup.sh\n\ [e2e-llm-inference-service] fi\n\nif [ \"$KSERVE_INFER_ROCE\" = \"true\" ]; then\n echo \"Trying to infer\ [e2e-llm-inference-service] \ RoCE configs ... \"\n grep -H . /sys/class/infiniband/*/ports/*/gids/* 2>/dev/null\n\ [e2e-llm-inference-service] \ grep -H . /sys/class/infiniband/*/ports/*/gid_attrs/types/* 2>/dev/null\n\ [e2e-llm-inference-service] \n cat /proc/driver/nvidia/params\n\n KSERVE_INFER_IB_GID_INDEX_GREP=${KSERVE_INFER_IB_GID_INDEX_GREP:-\"\ [e2e-llm-inference-service] RoCE v2\"}\n\n echo \"[Infer RoCE] Discovering active HCAs ...\"\n active_hcas=()\n\ [e2e-llm-inference-service] \ # Loop through all mlx5 devices found in sysfs\n for hca_dir in /sys/class/infiniband/mlx5_*;\ [e2e-llm-inference-service] \ do\n # Ensure it's a directory before proceeding\n if [ -d \"$hca_dir\"\ [e2e-llm-inference-service] \ ]; then\n hca_name=$(basename \"$hca_dir\")\n port_state_file=\"\ [e2e-llm-inference-service] $hca_dir/ports/1/state\" # Assume port 1\n type_file=\"$hca_dir/ports/1/gid_attrs/types/*\"\ [e2e-llm-inference-service] \n\n echo \"[Infer RoCE] Check if the port state file ${port_state_file}\ [e2e-llm-inference-service] \ exists and contains 'ACTIVE'\"\n if [ -f \"$port_state_file\" ] &&\ [e2e-llm-inference-service] \ grep -q \"ACTIVE\" \"$port_state_file\" && grep -q \"${KSERVE_INFER_IB_GID_INDEX_GREP}\"\ [e2e-llm-inference-service] \ ${type_file} 2>/dev/null; then\n echo \"[Infer RoCE] Found active\ [e2e-llm-inference-service] \ HCA: $hca_name\"\n active_hcas+=(\"$hca_name\")\n else\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Skipping inactive or down HCA: $hca_name\"\ [e2e-llm-inference-service] \n fi\n fi\n done\n\n # Check if we found any active HCAs\n\ [e2e-llm-inference-service] \ if [ ${#active_hcas[@]} -gt 0 ]; then\n # Join the array elements with\ [e2e-llm-inference-service] \ a comma\n hca_port_pairs=()\n for hca in \"${active_hcas[@]}\";\ [e2e-llm-inference-service] \ do\n hca_port_pairs+=(\"${hca}:1\")\n done\n\n active_hca_list=$(IFS=,;\ [e2e-llm-inference-service] \ echo \"${active_hcas[*]}\")\n hca_port_pairs_list=$(IFS=,; echo \"${hca_port_pairs[*]}\"\ [e2e-llm-inference-service] )\n echo \"[Infer RoCE] Setting active HCAs: ${active_hca_list}\"\n \ [e2e-llm-inference-service] \ export NCCL_IB_HCA=${NCCL_IB_HCA:-${active_hca_list}}\n export NVSHMEM_HCA_LIST=${NVSHMEM_HCA_LIST:-${hca_port_pairs_list}}\n\ [e2e-llm-inference-service] \ export UCX_NET_DEVICES=${UCX_NET_DEVICES:-${hca_port_pairs_list}}\n\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] NCCL_IB_HCA=${NCCL_IB_HCA}\"\n echo \"[Infer\ [e2e-llm-inference-service] \ RoCE] NVSHMEM_HCA_LIST=${NVSHMEM_HCA_LIST}\"\n echo \"[Infer RoCE] UCX_NET_DEVICES=${UCX_NET_DEVICES}\"\ [e2e-llm-inference-service] \n else\n echo \"[Infer RoCE] WARNING: No active RoCE HCAs found. NCCL_IB_HCA\ [e2e-llm-inference-service] \ will not be set.\"\n fi\n\n if [ ${#active_hcas[@]} -gt 0 ]; then\n \ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Finding GID_INDEX for each active HCA (SR-IOV compatible)...\"\ [e2e-llm-inference-service] \n\n # For SR-IOV environments, find the most common IPv4 RoCE v2 GID index\ [e2e-llm-inference-service] \ across all HCAs\n declare -A gid_index_count\n declare -A hca_gid_index\n\ [e2e-llm-inference-service] \n for hca_name in \"${active_hcas[@]}\"; do\n echo \"[Infer RoCE]\ [e2e-llm-inference-service] \ Processing HCA: ${hca_name}\"\n\n # Find all RoCE v2 IPv4 GIDs for\ [e2e-llm-inference-service] \ this HCA and count by index\n for tpath in /sys/class/infiniband/${hca_name}/ports/1/gid_attrs/types/*;\ [e2e-llm-inference-service] \ do\n if grep -q \"${KSERVE_INFER_IB_GID_INDEX_GREP}\" \"$tpath\"\ [e2e-llm-inference-service] \ 2>/dev/null; then\n idx=$(basename \"$tpath\")\n \ [e2e-llm-inference-service] \ gid_file=\"/sys/class/infiniband/${hca_name}/ports/1/gids/${idx}\"\ [e2e-llm-inference-service] \n # Check for IPv4 GID (contains ffff:)\n \ [e2e-llm-inference-service] \ if [ -f \"$gid_file\" ] && grep -q \"ffff:\" \"$gid_file\"; then\n \ [e2e-llm-inference-service] \ gid_value=$(cat \"$gid_file\" 2>/dev/null || echo \"\")\n \ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Found IPv4 RoCE v2 GID for ${hca_name}:\ [e2e-llm-inference-service] \ index=${idx}, gid=${gid_value}\"\n hca_gid_index[\"${hca_name}\"\ [e2e-llm-inference-service] ]=\"${idx}\"\n gid_index_count[\"${idx}\"]=$((${gid_index_count[\"\ [e2e-llm-inference-service] ${idx}\"]} + 1))\n break # Use first found IPv4 GID per\ [e2e-llm-inference-service] \ HCA\n fi\n fi\n done\n done\n\n\ [e2e-llm-inference-service] \ # Find the most common GID index (most likely to be consistent across\ [e2e-llm-inference-service] \ nodes)\n best_gid_index=\"\"\n max_count=0\n for idx in \"\ [e2e-llm-inference-service] ${!gid_index_count[@]}\"; do\n count=${gid_index_count[\"${idx}\"]}\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] GID_INDEX ${idx} found on ${count} HCAs\"\n \ [e2e-llm-inference-service] \ if [ $count -gt $max_count ]; then\n max_count=$count\n\ [e2e-llm-inference-service] \ best_gid_index=\"$idx\"\n fi\n done\n\n #\ [e2e-llm-inference-service] \ Use deterministic fallback if tied - prefer index 3 (SR-IOV standard)\n \ [e2e-llm-inference-service] \ if [ ${#gid_index_count[@]} -gt 1 ]; then\n echo \"[Infer RoCE]\ [e2e-llm-inference-service] \ Multiple GID indices found, selecting most common: ${best_gid_index}\"\n \ [e2e-llm-inference-service] \ # If there's a tie, prefer index 3 as it's most common in SR-IOV setups\n\ [e2e-llm-inference-service] \ if [ -n \"${gid_index_count['3']}\" ] && [ \"${gid_index_count['3']}\"\ [e2e-llm-inference-service] \ -eq \"$max_count\" ]; then\n best_gid_index=\"3\"\n \ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Using deterministic fallback: GID_INDEX=3 (SR-IOV\ [e2e-llm-inference-service] \ standard)\"\n fi\n fi\n\n # Check if GID_INDEX is already\ [e2e-llm-inference-service] \ set via environment variables\n if [ -n \"${NCCL_IB_GID_INDEX}\" ]; then\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Using pre-configured NCCL_IB_GID_INDEX=${NCCL_IB_GID_INDEX}\ [e2e-llm-inference-service] \ from environment\"\n export NVSHMEM_IB_GID_INDEX=${NVSHMEM_IB_GID_INDEX:-$NCCL_IB_GID_INDEX}\n\ [e2e-llm-inference-service] \ export UCX_IB_GID_INDEX=${UCX_IB_GID_INDEX:-$NCCL_IB_GID_INDEX}\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Using pre-configured GID_INDEX=${NCCL_IB_GID_INDEX}\ [e2e-llm-inference-service] \ for NCCL, NVSHMEM, and UCX\"\n elif [ -n \"$best_gid_index\" ]; then\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Selected GID_INDEX: ${best_gid_index} (found\ [e2e-llm-inference-service] \ on ${max_count} HCAs)\"\n\n export NCCL_IB_GID_INDEX=${NCCL_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \ export NVSHMEM_IB_GID_INDEX=${NVSHMEM_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \ export UCX_IB_GID_INDEX=${UCX_IB_GID_INDEX:-$best_gid_index}\n\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Exported GID_INDEX=${best_gid_index} for NCCL,\ [e2e-llm-inference-service] \ NVSHMEM, and UCX\"\n else\n echo \"[Infer RoCE] ERROR: No valid\ [e2e-llm-inference-service] \ IPv4 ${KSERVE_INFER_IB_GID_INDEX_GREP} GID_INDEX found on any HCA.\"\n \ [e2e-llm-inference-service] \ fi\n else\n echo \"[Infer RoCE] No active HCAs found, skipping GID_INDEX\ [e2e-llm-inference-service] \ inference.\"\n fi\nfi\n\n# --disable-access-log-for-endpoints landed in vLLM\ [e2e-llm-inference-service] \ 0.16.0 (vllm-project/vllm#30011).\n# Older versions still need the blanket\ [e2e-llm-inference-service] \ --disable-uvicorn-access-log.\nACCESS_LOG_ARGS=\"--disable-uvicorn-access-log\"\ [e2e-llm-inference-service] \nVLLM_VERSION=$(vllm --version 2>/dev/null | tail -1 | awk '{print $NF}')\n\ [e2e-llm-inference-service] echo \"[access-log-detect] vllm version='${VLLM_VERSION}'\"\nif [[ \"$VLLM_VERSION\"\ [e2e-llm-inference-service] \ =~ ^[0-9]+\\.[0-9]+ ]] && [ \"$(printf '%s\\n%s\\n' \"0.16.0\" \"${VLLM_VERSION}\"\ [e2e-llm-inference-service] \ | sort -V | head -1)\" = \"0.16.0\" ]; then\n ACCESS_LOG_ARGS=\"--disable-access-log-for-endpoints\ [e2e-llm-inference-service] \ /health,/metrics,/ping\"\nfi\necho \"[access-log-detect] selected ACCESS_LOG_ARGS='${ACCESS_LOG_ARGS}'\"\ [e2e-llm-inference-service] \n\n# --shutdown-timeout landed in vLLM 0.18.0 (vllm-project/vllm#36666).\n\ [e2e-llm-inference-service] SHUTDOWN_TIMEOUT_ARGS=\"\"\nif [[ \"$VLLM_VERSION\" =~ ^[0-9]+\\.[0-9]+ ]] &&\ [e2e-llm-inference-service] \ [ \"$(printf '%s\\n%s\\n' \"0.18.0\" \"${VLLM_VERSION}\" | sort -V | head\ [e2e-llm-inference-service] \ -1)\" = \"0.18.0\" ]; then\n SHUTDOWN_TIMEOUT_ARGS=\"--shutdown-timeout 40\"\ [e2e-llm-inference-service] \nfi\n\n# --kv-transfer-config with OffloadingConnector requires vLLM 0.22.0+\ [e2e-llm-inference-service] \ (vllm-project/vllm#40020).\nKV_TRANSFER_ARGS=\"\"\nif [[ \"$VLLM_VERSION\"\ [e2e-llm-inference-service] \ =~ ^[0-9]+\\.[0-9]+ ]] && [ \"$(printf '%s\\n%s\\n' \"0.22.0\" \"${VLLM_VERSION}\"\ [e2e-llm-inference-service] \ | sort -V | head -1)\" = \"0.22.0\" ]; then\n if [[ \"${VLLM_ADDITIONAL_ARGS:-}\"\ [e2e-llm-inference-service] \ != *\"--kv-transfer-config\"* ]] && [[ \"${VLLM_ADDITIONAL_ARGS:-}\" != *\"\ [e2e-llm-inference-service] --kv_transfer_config\"* ]] && [[ \"$*\" != *\"--kv-transfer-config\"* ]] &&\ [e2e-llm-inference-service] \ [[ \"$*\" != *\"--kv_transfer_config\"* ]]; then\n KV_TRANSFER_ARGS=\"\"\ [e2e-llm-inference-service] \n fi\nfi\n\neval \"exec vllm serve /mnt/models \\\n --served-model-name \"\ [e2e-llm-inference-service] facebook/opt-125m\" \"publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m\"\ [e2e-llm-inference-service] \ \\\n --port 8000 \\\n ${ACCESS_LOG_ARGS} \\\n ${SHUTDOWN_TIMEOUT_ARGS}\ [e2e-llm-inference-service] \ \\\n ${KV_TRANSFER_ARGS} \\\n \\\n --enable-ssl-refresh \\\n --ssl-certfile\ [e2e-llm-inference-service] \ /var/run/kserve/tls/tls.crt \\\n --ssl-keyfile /var/run/kserve/tls/tls.key\ [e2e-llm-inference-service] \ \\\n ${VLLM_ADDITIONAL_ARGS} \\\n $@\"" [e2e-llm-inference-service] - -- [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: DEBUG [e2e-llm-inference-service] - name: VLLM_CPU_KVCACHE_SPACE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: VLLM_ENABLE_V1_MULTIPROCESSING [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: USER [e2e-llm-inference-service] value: nonroot [e2e-llm-inference-service] - name: TORCHINDUCTOR_CACHE_DIR [e2e-llm-inference-service] value: /tmp/torchinductor-cache [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '2' [e2e-llm-inference-service] memory: 7Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] - name: kube-api-access-hcmjt [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000780000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: false [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: default [e2e-llm-inference-service] serviceAccount: default [e2e-llm-inference-service] nodeName: ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c28,c12 [e2e-llm-inference-service] fsGroup: 1000780000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-wbrnh [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:07Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] hostIP: 10.0.138.196 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.138.196 [e2e-llm-inference-service] podIP: 10.134.0.34 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.134.0.34 [e2e-llm-inference-service] startTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] initContainerStatuses: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] state: [e2e-llm-inference-service] terminated: [e2e-llm-inference-service] exitCode: 0 [e2e-llm-inference-service] reason: Completed [e2e-llm-inference-service] startedAt: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] finishedAt: '2026-07-20T10:14:07Z' [e2e-llm-inference-service] containerID: cri-o://407b1b1be7288044f29990209cffff8a92fdf45c165adc56360b201c0df48cdf [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] imageID: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] containerID: cri-o://407b1b1be7288044f29990209cffff8a92fdf45c165adc56360b201c0df48cdf [e2e-llm-inference-service] started: false [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] - name: kube-api-access-hcmjt [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000780000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000780000 [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-20T10:14:38Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0 [e2e-llm-inference-service] imageID: public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo@sha256:afb39fca138b51d019d986229d546531b45a2a3deb73bcf59bd42406e13fbba0 [e2e-llm-inference-service] containerID: cri-o://3dbd39decd1dcf27f61d652d724eecaf2b3e2f565482a65b7254a8393a6ab090 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '2' [e2e-llm-inference-service] memory: 7Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: kube-api-access-hcmjt [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000780000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000780000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler-794498b458-mt7dc [e2e-llm-inference-service] generateName: auth-enabled-test-kserve-router-scheduler-794498b458- [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: b99ce9e9-0067-46b0-a74e-9b05501a3c02 [e2e-llm-inference-service] resourceVersion: '27158' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 794498b458 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.134.0.35/23"],"mac_address":"0a:58:0a:86:00:23","gateway_ips":["10.134.0.1"],"routes":[{"dest":"10.132.0.0/14","nextHop":"10.134.0.1"},{"dest":"172.31.0.0/16","nextHop":"10.134.0.1"},{"dest":"169.254.0.5/32","nextHop":"10.134.0.1"},{"dest":"100.64.0.0/16","nextHop":"10.134.0.1"}],"ip_address":"10.134.0.35/23","gateway_ip":"10.134.0.1","role":"primary"}}' [e2e-llm-inference-service] k8s.v1.cni.cncf.io/network-status: "[{\n \"name\": \"ovn-kubernetes\",\n \ [e2e-llm-inference-service] \ \"interface\": \"eth0\",\n \"ips\": [\n \"10.134.0.35\"\n ],\n\ [e2e-llm-inference-service] \ \"mac\": \"0a:58:0a:86:00:23\",\n \"default\": true,\n \"dns\": {}\n\ [e2e-llm-inference-service] }]" [e2e-llm-inference-service] openshift.io/scc: restricted-v2 [e2e-llm-inference-service] seccomp.security.alpha.kubernetes.io/pod: runtime/default [e2e-llm-inference-service] security.openshift.io/validated-scc-subject-type: user [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler-794498b458 [e2e-llm-inference-service] uid: 7794b4d5-fa45-48ea-877e-68aef52cbdec [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: ip-10-0-138-196 [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.ovn.org/pod-networks: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"7794b4d5-fa45-48ea-877e-68aef52cbdec"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:enableServiceLinks: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"storage-initializer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"AWS_ACCESS_KEY_ID"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE_CONFIGMAP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_ENDPOINT_URL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_SECRET_ACCESS_KEY"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_ENABLE_HF_TRANSFER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_HIGH_PERFORMANCE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_NUM_CONCURRENT_RANGE_GETS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_ENDPOINT"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_USE_HTTPS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_VERIFY_SSL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"STORAGE_ALLOW_PATTERNS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"kserve-provision-location"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: multus-daemon [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:k8s.v1.cni.cncf.io/network-status: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] - manager: kubelet [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] k:{"type":"ContainersReady"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Initialized"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodReadyToStartContainers"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"PodScheduled"}: [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] k:{"type":"Ready"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastProbeTime: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:containerStatuses: {} [e2e-llm-inference-service] f:hostIP: {} [e2e-llm-inference-service] f:hostIPs: {} [e2e-llm-inference-service] f:initContainerStatuses: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:phase: {} [e2e-llm-inference-service] f:podIP: {} [e2e-llm-inference-service] f:podIPs: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"ip":"10.134.0.35"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:ip: {} [e2e-llm-inference-service] f:startTime: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: kube-api-access-xdjbz [e2e-llm-inference-service] projected: [e2e-llm-inference-service] sources: [e2e-llm-inference-service] - serviceAccountToken: [e2e-llm-inference-service] expirationSeconds: 3607 [e2e-llm-inference-service] path: token [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: kube-root-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: ca.crt [e2e-llm-inference-service] path: ca.crt [e2e-llm-inference-service] - downwardAPI: [e2e-llm-inference-service] items: [e2e-llm-inference-service] - path: namespace [e2e-llm-inference-service] fieldRef: [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] fieldPath: metadata.namespace [e2e-llm-inference-service] - configMap: [e2e-llm-inference-service] name: openshift-service-ca.crt [e2e-llm-inference-service] items: [e2e-llm-inference-service] - key: service-ca.crt [e2e-llm-inference-service] path: service-ca.crt [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] args: [e2e-llm-inference-service] - hf://facebook/opt-125m [e2e-llm-inference-service] - /mnt/models [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] - name: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] - name: S3_USE_HTTPS [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: S3_ENDPOINT [e2e-llm-inference-service] value: s3-service.kserve:8333 [e2e-llm-inference-service] - name: AWS_ENDPOINT_URL [e2e-llm-inference-service] value: http://s3-service.kserve:8333 [e2e-llm-inference-service] - name: S3_VERIFY_SSL [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: AWS_CA_BUNDLE [e2e-llm-inference-service] value: /etc/ssl/custom-certs/cabundle.crt [e2e-llm-inference-service] - name: AWS_CA_BUNDLE_CONFIGMAP [e2e-llm-inference-service] value: odh-kserve-custom-ca-bundle [e2e-llm-inference-service] - name: HF_HUB_ENABLE_HF_TRANSFER [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_HIGH_PERFORMANCE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_NUM_CONCURRENT_RANGE_GETS [e2e-llm-inference-service] value: '8' [e2e-llm-inference-service] - name: STORAGE_ALLOW_PATTERNS [e2e-llm-inference-service] value: '["tokenizer.json", "tokenizer_config.json", "special_tokens_map.json", [e2e-llm-inference-service] "vocab.json", "merges.txt", "config.json", "generation_config.json"]' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] - name: kube-api-access-xdjbz [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000780000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - auth-enabled-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: single-profile-handler\n- type: queue-scorer\n- type: kv-cache-utilization-scorer\n\ [e2e-llm-inference-service] - type: prefix-cache-scorer\n- type: no-hit-lru-scorer\n- type: max-score-picker\n\ [e2e-llm-inference-service] schedulingProfiles:\n- name: default\n plugins:\n - pluginRef: queue-scorer\n\ [e2e-llm-inference-service] \ weight: 2\n - pluginRef: kv-cache-utilization-scorer\n weight: 2\n\ [e2e-llm-inference-service] \ - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef: no-hit-lru-scorer\n\ [e2e-llm-inference-service] \ weight: 2\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '6' [e2e-llm-inference-service] memory: 16Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] - name: kube-api-access-xdjbz [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsUser: 1000780000 [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: auth-enabled-test-epp-sa [e2e-llm-inference-service] serviceAccount: auth-enabled-test-epp-sa [e2e-llm-inference-service] nodeName: ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] seLinuxOptions: [e2e-llm-inference-service] level: s0:c28,c12 [e2e-llm-inference-service] fsGroup: 1000780000 [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-wbrnh [e2e-llm-inference-service] - name: auth-enabled-test-epp-sa-dockercfg-q7z5g [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] tolerations: [e2e-llm-inference-service] - key: node.kubernetes.io/not-ready [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/unreachable [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoExecute [e2e-llm-inference-service] tolerationSeconds: 300 [e2e-llm-inference-service] - key: node.kubernetes.io/memory-pressure [e2e-llm-inference-service] operator: Exists [e2e-llm-inference-service] effect: NoSchedule [e2e-llm-inference-service] priority: 0 [e2e-llm-inference-service] enableServiceLinks: true [e2e-llm-inference-service] preemptionPolicy: PreemptLowerPriority [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] phase: Running [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: PodReadyToStartContainers [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] - type: Initialized [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:05Z' [e2e-llm-inference-service] - type: Ready [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] - type: ContainersReady [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] - type: PodScheduled [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastProbeTime: null [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] hostIP: 10.0.138.196 [e2e-llm-inference-service] hostIPs: [e2e-llm-inference-service] - ip: 10.0.138.196 [e2e-llm-inference-service] podIP: 10.134.0.35 [e2e-llm-inference-service] podIPs: [e2e-llm-inference-service] - ip: 10.134.0.35 [e2e-llm-inference-service] startTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] initContainerStatuses: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] state: [e2e-llm-inference-service] terminated: [e2e-llm-inference-service] exitCode: 0 [e2e-llm-inference-service] reason: Completed [e2e-llm-inference-service] startedAt: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] finishedAt: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] containerID: cri-o://91f5006d7c75b8898a20f12d2451c53210309ba663126fd2bd198966fa8d07da [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] imageID: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] containerID: cri-o://91f5006d7c75b8898a20f12d2451c53210309ba663126fd2bd198966fa8d07da [e2e-llm-inference-service] started: false [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] - name: kube-api-access-xdjbz [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000780000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000780000 [e2e-llm-inference-service] containerStatuses: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] state: [e2e-llm-inference-service] running: [e2e-llm-inference-service] startedAt: '2026-07-20T10:14:05Z' [e2e-llm-inference-service] lastState: {} [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] restartCount: 0 [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] imageID: quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:7fd1d61cb5505e026eebab0b9653758779c3dee241e37bea92a0bab961a48c50 [e2e-llm-inference-service] containerID: cri-o://64568097fca2d9c1da0e8b812fed17dd838a87ce73d19005684bf7cfa7d68789 [e2e-llm-inference-service] started: true [e2e-llm-inference-service] allocatedResources: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '6' [e2e-llm-inference-service] memory: 16Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] - name: kube-api-access-xdjbz [e2e-llm-inference-service] mountPath: /var/run/secrets/kubernetes.io/serviceaccount [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] recursiveReadOnly: Disabled [e2e-llm-inference-service] user: [e2e-llm-inference-service] linux: [e2e-llm-inference-service] uid: 1000780000 [e2e-llm-inference-service] gid: 0 [e2e-llm-inference-service] supplementalGroups: [e2e-llm-inference-service] - 0 [e2e-llm-inference-service] - 1000780000 [e2e-llm-inference-service] qosClass: Burstable [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-sa [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 9736689d-adab-488e-84d0-a72c2540cea9 [e2e-llm-inference-service] resourceVersion: '26153' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] openshift.io/internal-registry-pull-secret-ref: auth-enabled-test-epp-sa-dockercfg-q7z5g [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: openshift.io/image-registry-pull-secrets_service-account-controller [e2e-llm-inference-service] operation: Apply [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:imagePullSecrets: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] f:openshift.io/internal-registry-pull-secret-ref: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] k:{"name":"auth-enabled-test-epp-sa-dockercfg-q7z5g"}: {} [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:secrets: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"default-dockercfg-wbrnh"}: {} [e2e-llm-inference-service] k:{"name":"seaweedfs-s3-creds"}: {} [e2e-llm-inference-service] secrets: [e2e-llm-inference-service] - name: seaweedfs-s3-creds [e2e-llm-inference-service] - name: default-dockercfg-wbrnh [e2e-llm-inference-service] - name: auth-enabled-test-epp-sa-dockercfg-q7z5g [e2e-llm-inference-service] imagePullSecrets: [e2e-llm-inference-service] - name: default-dockercfg-wbrnh [e2e-llm-inference-service] - name: auth-enabled-test-epp-sa-dockercfg-q7z5g [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: ServiceAccount [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-service [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 4c53d392-887a-412f-9eef-69b74e71136c [e2e-llm-inference-service] resourceVersion: '26169' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:internalTrafficPolicy: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"port":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] k:{"port":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:sessionAffinity: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] spec: [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] targetPort: grpc [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] targetPort: grpc-health [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] targetPort: metrics [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] targetPort: zmq [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] clusterIP: 172.31.29.235 [e2e-llm-inference-service] clusterIPs: [e2e-llm-inference-service] - 172.31.29.235 [e2e-llm-inference-service] type: ClusterIP [e2e-llm-inference-service] sessionAffinity: None [e2e-llm-inference-service] ipFamilies: [e2e-llm-inference-service] - IPv4 [e2e-llm-inference-service] ipFamilyPolicy: SingleStack [e2e-llm-inference-service] internalTrafficPolicy: Cluster [e2e-llm-inference-service] status: [e2e-llm-inference-service] loadBalancer: {} [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 0cb27ec7-b426-439d-93b3-877bb0444800 [e2e-llm-inference-service] resourceVersion: '26141' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:internalTrafficPolicy: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"port":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:appProtocol: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:targetPort: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:sessionAffinity: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] spec: [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] targetPort: 8000 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] clusterIP: 172.31.78.75 [e2e-llm-inference-service] clusterIPs: [e2e-llm-inference-service] - 172.31.78.75 [e2e-llm-inference-service] type: ClusterIP [e2e-llm-inference-service] sessionAffinity: None [e2e-llm-inference-service] ipFamilies: [e2e-llm-inference-service] - IPv4 [e2e-llm-inference-service] ipFamilyPolicy: SingleStack [e2e-llm-inference-service] internalTrafficPolicy: Cluster [e2e-llm-inference-service] status: [e2e-llm-inference-service] loadBalancer: {} [e2e-llm-inference-service] apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 6d81965c-bb64-4052-8e35-8d58e0e45bf4 [e2e-llm-inference-service] resourceVersion: '28785' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:rollingUpdate: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:maxSurge: {} [e2e-llm-inference-service] f:maxUnavailable: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"TORCHINDUCTOR_CACHE_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"USER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_CPU_KVCACHE_SPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_ENABLE_V1_MULTIPROCESSING"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"storage-initializer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"AWS_ACCESS_KEY_ID"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE_CONFIGMAP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_ENDPOINT_URL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_SECRET_ACCESS_KEY"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_ENABLE_HF_TRANSFER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_HIGH_PERFORMANCE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_NUM_CONCURRENT_RANGE_GETS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_ENDPOINT"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_USE_HTTPS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_VERIFY_SSL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"TOKIO_WORKER_THREADS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"kserve-provision-location"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] args: [e2e-llm-inference-service] - hf://facebook/opt-125m [e2e-llm-inference-service] - /mnt/models [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: TOKIO_WORKER_THREADS [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] - name: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] - name: S3_USE_HTTPS [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: S3_ENDPOINT [e2e-llm-inference-service] value: s3-service.kserve:8333 [e2e-llm-inference-service] - name: AWS_ENDPOINT_URL [e2e-llm-inference-service] value: http://s3-service.kserve:8333 [e2e-llm-inference-service] - name: S3_VERIFY_SSL [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: AWS_CA_BUNDLE [e2e-llm-inference-service] value: /etc/ssl/custom-certs/cabundle.crt [e2e-llm-inference-service] - name: AWS_CA_BUNDLE_CONFIGMAP [e2e-llm-inference-service] value: odh-kserve-custom-ca-bundle [e2e-llm-inference-service] - name: HF_HUB_ENABLE_HF_TRANSFER [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_HIGH_PERFORMANCE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_NUM_CONCURRENT_RANGE_GETS [e2e-llm-inference-service] value: '8' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/bash [e2e-llm-inference-service] - -c [e2e-llm-inference-service] - "if [ -f /etc/profile.d/ibm-aiu-setup.sh ]; then\n source /etc/profile.d/ibm-aiu-setup.sh\n\ [e2e-llm-inference-service] fi\n\nif [ \"$KSERVE_INFER_ROCE\" = \"true\" ]; then\n echo \"Trying to\ [e2e-llm-inference-service] \ infer RoCE configs ... \"\n grep -H . /sys/class/infiniband/*/ports/*/gids/*\ [e2e-llm-inference-service] \ 2>/dev/null\n grep -H . /sys/class/infiniband/*/ports/*/gid_attrs/types/*\ [e2e-llm-inference-service] \ 2>/dev/null\n\n cat /proc/driver/nvidia/params\n\n KSERVE_INFER_IB_GID_INDEX_GREP=${KSERVE_INFER_IB_GID_INDEX_GREP:-\"\ [e2e-llm-inference-service] RoCE v2\"}\n\n echo \"[Infer RoCE] Discovering active HCAs ...\"\n active_hcas=()\n\ [e2e-llm-inference-service] \ # Loop through all mlx5 devices found in sysfs\n for hca_dir in /sys/class/infiniband/mlx5_*;\ [e2e-llm-inference-service] \ do\n # Ensure it's a directory before proceeding\n if [ -d \"\ [e2e-llm-inference-service] $hca_dir\" ]; then\n hca_name=$(basename \"$hca_dir\")\n \ [e2e-llm-inference-service] \ port_state_file=\"$hca_dir/ports/1/state\" # Assume port 1\n \ [e2e-llm-inference-service] \ type_file=\"$hca_dir/ports/1/gid_attrs/types/*\"\n\n echo\ [e2e-llm-inference-service] \ \"[Infer RoCE] Check if the port state file ${port_state_file} exists\ [e2e-llm-inference-service] \ and contains 'ACTIVE'\"\n if [ -f \"$port_state_file\" ] && grep\ [e2e-llm-inference-service] \ -q \"ACTIVE\" \"$port_state_file\" && grep -q \"${KSERVE_INFER_IB_GID_INDEX_GREP}\"\ [e2e-llm-inference-service] \ ${type_file} 2>/dev/null; then\n echo \"[Infer RoCE] Found\ [e2e-llm-inference-service] \ active HCA: $hca_name\"\n active_hcas+=(\"$hca_name\")\n\ [e2e-llm-inference-service] \ else\n echo \"[Infer RoCE] Skipping inactive or\ [e2e-llm-inference-service] \ down HCA: $hca_name\"\n fi\n fi\n done\n\n # Check if\ [e2e-llm-inference-service] \ we found any active HCAs\n if [ ${#active_hcas[@]} -gt 0 ]; then\n \ [e2e-llm-inference-service] \ # Join the array elements with a comma\n hca_port_pairs=()\n \ [e2e-llm-inference-service] \ for hca in \"${active_hcas[@]}\"; do\n hca_port_pairs+=(\"\ [e2e-llm-inference-service] ${hca}:1\")\n done\n\n active_hca_list=$(IFS=,; echo \"${active_hcas[*]}\"\ [e2e-llm-inference-service] )\n hca_port_pairs_list=$(IFS=,; echo \"${hca_port_pairs[*]}\")\n \ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Setting active HCAs: ${active_hca_list}\"\n \ [e2e-llm-inference-service] \ export NCCL_IB_HCA=${NCCL_IB_HCA:-${active_hca_list}}\n export\ [e2e-llm-inference-service] \ NVSHMEM_HCA_LIST=${NVSHMEM_HCA_LIST:-${hca_port_pairs_list}}\n export\ [e2e-llm-inference-service] \ UCX_NET_DEVICES=${UCX_NET_DEVICES:-${hca_port_pairs_list}}\n\n echo\ [e2e-llm-inference-service] \ \"[Infer RoCE] NCCL_IB_HCA=${NCCL_IB_HCA}\"\n echo \"[Infer RoCE]\ [e2e-llm-inference-service] \ NVSHMEM_HCA_LIST=${NVSHMEM_HCA_LIST}\"\n echo \"[Infer RoCE] UCX_NET_DEVICES=${UCX_NET_DEVICES}\"\ [e2e-llm-inference-service] \n else\n echo \"[Infer RoCE] WARNING: No active RoCE HCAs found.\ [e2e-llm-inference-service] \ NCCL_IB_HCA will not be set.\"\n fi\n\n if [ ${#active_hcas[@]} -gt\ [e2e-llm-inference-service] \ 0 ]; then\n echo \"[Infer RoCE] Finding GID_INDEX for each active\ [e2e-llm-inference-service] \ HCA (SR-IOV compatible)...\"\n\n # For SR-IOV environments, find\ [e2e-llm-inference-service] \ the most common IPv4 RoCE v2 GID index across all HCAs\n declare\ [e2e-llm-inference-service] \ -A gid_index_count\n declare -A hca_gid_index\n\n for hca_name\ [e2e-llm-inference-service] \ in \"${active_hcas[@]}\"; do\n echo \"[Infer RoCE] Processing\ [e2e-llm-inference-service] \ HCA: ${hca_name}\"\n\n # Find all RoCE v2 IPv4 GIDs for this\ [e2e-llm-inference-service] \ HCA and count by index\n for tpath in /sys/class/infiniband/${hca_name}/ports/1/gid_attrs/types/*;\ [e2e-llm-inference-service] \ do\n if grep -q \"${KSERVE_INFER_IB_GID_INDEX_GREP}\" \"\ [e2e-llm-inference-service] $tpath\" 2>/dev/null; then\n idx=$(basename \"$tpath\"\ [e2e-llm-inference-service] )\n gid_file=\"/sys/class/infiniband/${hca_name}/ports/1/gids/${idx}\"\ [e2e-llm-inference-service] \n # Check for IPv4 GID (contains ffff:)\n \ [e2e-llm-inference-service] \ if [ -f \"$gid_file\" ] && grep -q \"ffff:\" \"$gid_file\"; then\n\ [e2e-llm-inference-service] \ gid_value=$(cat \"$gid_file\" 2>/dev/null || echo\ [e2e-llm-inference-service] \ \"\")\n echo \"[Infer RoCE] Found IPv4 RoCE v2 GID\ [e2e-llm-inference-service] \ for ${hca_name}: index=${idx}, gid=${gid_value}\"\n \ [e2e-llm-inference-service] \ hca_gid_index[\"${hca_name}\"]=\"${idx}\"\n gid_index_count[\"\ [e2e-llm-inference-service] ${idx}\"]=$((${gid_index_count[\"${idx}\"]} + 1))\n \ [e2e-llm-inference-service] \ break # Use first found IPv4 GID per HCA\n fi\n \ [e2e-llm-inference-service] \ fi\n done\n done\n\n # Find the most common\ [e2e-llm-inference-service] \ GID index (most likely to be consistent across nodes)\n best_gid_index=\"\ [e2e-llm-inference-service] \"\n max_count=0\n for idx in \"${!gid_index_count[@]}\"; do\n\ [e2e-llm-inference-service] \ count=${gid_index_count[\"${idx}\"]}\n echo \"[Infer\ [e2e-llm-inference-service] \ RoCE] GID_INDEX ${idx} found on ${count} HCAs\"\n if [ $count\ [e2e-llm-inference-service] \ -gt $max_count ]; then\n max_count=$count\n \ [e2e-llm-inference-service] \ best_gid_index=\"$idx\"\n fi\n done\n\n # Use deterministic\ [e2e-llm-inference-service] \ fallback if tied - prefer index 3 (SR-IOV standard)\n if [ ${#gid_index_count[@]}\ [e2e-llm-inference-service] \ -gt 1 ]; then\n echo \"[Infer RoCE] Multiple GID indices found,\ [e2e-llm-inference-service] \ selecting most common: ${best_gid_index}\"\n # If there's a tie,\ [e2e-llm-inference-service] \ prefer index 3 as it's most common in SR-IOV setups\n if [ -n\ [e2e-llm-inference-service] \ \"${gid_index_count['3']}\" ] && [ \"${gid_index_count['3']}\" -eq \"\ [e2e-llm-inference-service] $max_count\" ]; then\n best_gid_index=\"3\"\n \ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Using deterministic fallback: GID_INDEX=3 (SR-IOV\ [e2e-llm-inference-service] \ standard)\"\n fi\n fi\n\n # Check if GID_INDEX is already\ [e2e-llm-inference-service] \ set via environment variables\n if [ -n \"${NCCL_IB_GID_INDEX}\"\ [e2e-llm-inference-service] \ ]; then\n echo \"[Infer RoCE] Using pre-configured NCCL_IB_GID_INDEX=${NCCL_IB_GID_INDEX}\ [e2e-llm-inference-service] \ from environment\"\n export NVSHMEM_IB_GID_INDEX=${NVSHMEM_IB_GID_INDEX:-$NCCL_IB_GID_INDEX}\n\ [e2e-llm-inference-service] \ export UCX_IB_GID_INDEX=${UCX_IB_GID_INDEX:-$NCCL_IB_GID_INDEX}\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Using pre-configured GID_INDEX=${NCCL_IB_GID_INDEX}\ [e2e-llm-inference-service] \ for NCCL, NVSHMEM, and UCX\"\n elif [ -n \"$best_gid_index\" ]; then\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Selected GID_INDEX: ${best_gid_index} (found\ [e2e-llm-inference-service] \ on ${max_count} HCAs)\"\n\n export NCCL_IB_GID_INDEX=${NCCL_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \ export NVSHMEM_IB_GID_INDEX=${NVSHMEM_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \ export UCX_IB_GID_INDEX=${UCX_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \n echo \"[Infer RoCE] Exported GID_INDEX=${best_gid_index} for\ [e2e-llm-inference-service] \ NCCL, NVSHMEM, and UCX\"\n else\n echo \"[Infer RoCE] ERROR:\ [e2e-llm-inference-service] \ No valid IPv4 ${KSERVE_INFER_IB_GID_INDEX_GREP} GID_INDEX found on any\ [e2e-llm-inference-service] \ HCA.\"\n fi\n else\n echo \"[Infer RoCE] No active HCAs found,\ [e2e-llm-inference-service] \ skipping GID_INDEX inference.\"\n fi\nfi\n\n# --disable-access-log-for-endpoints\ [e2e-llm-inference-service] \ landed in vLLM 0.16.0 (vllm-project/vllm#30011).\n# Older versions still\ [e2e-llm-inference-service] \ need the blanket --disable-uvicorn-access-log.\nACCESS_LOG_ARGS=\"--disable-uvicorn-access-log\"\ [e2e-llm-inference-service] \nVLLM_VERSION=$(vllm --version 2>/dev/null | tail -1 | awk '{print $NF}')\n\ [e2e-llm-inference-service] echo \"[access-log-detect] vllm version='${VLLM_VERSION}'\"\nif [[ \"$VLLM_VERSION\"\ [e2e-llm-inference-service] \ =~ ^[0-9]+\\.[0-9]+ ]] && [ \"$(printf '%s\\n%s\\n' \"0.16.0\" \"${VLLM_VERSION}\"\ [e2e-llm-inference-service] \ | sort -V | head -1)\" = \"0.16.0\" ]; then\n ACCESS_LOG_ARGS=\"--disable-access-log-for-endpoints\ [e2e-llm-inference-service] \ /health,/metrics,/ping\"\nfi\necho \"[access-log-detect] selected ACCESS_LOG_ARGS='${ACCESS_LOG_ARGS}'\"\ [e2e-llm-inference-service] \n\n# --shutdown-timeout landed in vLLM 0.18.0 (vllm-project/vllm#36666).\n\ [e2e-llm-inference-service] SHUTDOWN_TIMEOUT_ARGS=\"\"\nif [[ \"$VLLM_VERSION\" =~ ^[0-9]+\\.[0-9]+\ [e2e-llm-inference-service] \ ]] && [ \"$(printf '%s\\n%s\\n' \"0.18.0\" \"${VLLM_VERSION}\" | sort\ [e2e-llm-inference-service] \ -V | head -1)\" = \"0.18.0\" ]; then\n SHUTDOWN_TIMEOUT_ARGS=\"--shutdown-timeout\ [e2e-llm-inference-service] \ 40\"\nfi\n\n# --kv-transfer-config with OffloadingConnector requires vLLM\ [e2e-llm-inference-service] \ 0.22.0+ (vllm-project/vllm#40020).\nKV_TRANSFER_ARGS=\"\"\nif [[ \"$VLLM_VERSION\"\ [e2e-llm-inference-service] \ =~ ^[0-9]+\\.[0-9]+ ]] && [ \"$(printf '%s\\n%s\\n' \"0.22.0\" \"${VLLM_VERSION}\"\ [e2e-llm-inference-service] \ | sort -V | head -1)\" = \"0.22.0\" ]; then\n if [[ \"${VLLM_ADDITIONAL_ARGS:-}\"\ [e2e-llm-inference-service] \ != *\"--kv-transfer-config\"* ]] && [[ \"${VLLM_ADDITIONAL_ARGS:-}\" !=\ [e2e-llm-inference-service] \ *\"--kv_transfer_config\"* ]] && [[ \"$*\" != *\"--kv-transfer-config\"\ [e2e-llm-inference-service] * ]] && [[ \"$*\" != *\"--kv_transfer_config\"* ]]; then\n KV_TRANSFER_ARGS=\"\ [e2e-llm-inference-service] \"\n fi\nfi\n\neval \"exec vllm serve /mnt/models \\\n --served-model-name\ [e2e-llm-inference-service] \ \"facebook/opt-125m\" \"publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m\"\ [e2e-llm-inference-service] \ \\\n --port 8000 \\\n ${ACCESS_LOG_ARGS} \\\n ${SHUTDOWN_TIMEOUT_ARGS}\ [e2e-llm-inference-service] \ \\\n ${KV_TRANSFER_ARGS} \\\n \\\n --enable-ssl-refresh \\\n --ssl-certfile\ [e2e-llm-inference-service] \ /var/run/kserve/tls/tls.crt \\\n --ssl-keyfile /var/run/kserve/tls/tls.key\ [e2e-llm-inference-service] \ \\\n ${VLLM_ADDITIONAL_ARGS} \\\n $@\"" [e2e-llm-inference-service] - -- [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: DEBUG [e2e-llm-inference-service] - name: VLLM_CPU_KVCACHE_SPACE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: VLLM_ENABLE_V1_MULTIPROCESSING [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: USER [e2e-llm-inference-service] value: nonroot [e2e-llm-inference-service] - name: TORCHINDUCTOR_CACHE_DIR [e2e-llm-inference-service] value: /tmp/torchinductor-cache [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '2' [e2e-llm-inference-service] memory: 7Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: false [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: RollingUpdate [e2e-llm-inference-service] rollingUpdate: [e2e-llm-inference-service] maxUnavailable: 25% [e2e-llm-inference-service] maxSurge: 25% [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "auth-enabled-test-kserve-6c997987dd" has successfully progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 8c613b3a-ecbb-4e4d-8a9a-2633ca7781a3 [e2e-llm-inference-service] resourceVersion: '27163' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:progressDeadlineSeconds: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:revisionHistoryLimit: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:strategy: [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"storage-initializer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"AWS_ACCESS_KEY_ID"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE_CONFIGMAP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_ENDPOINT_URL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_SECRET_ACCESS_KEY"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_ENABLE_HF_TRANSFER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_HIGH_PERFORMANCE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_NUM_CONCURRENT_RANGE_GETS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_ENDPOINT"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_USE_HTTPS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_VERIFY_SSL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"STORAGE_ALLOW_PATTERNS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"kserve-provision-location"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:conditions: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"type":"Available"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] k:{"type":"Progressing"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:lastTransitionTime: {} [e2e-llm-inference-service] f:lastUpdateTime: {} [e2e-llm-inference-service] f:message: {} [e2e-llm-inference-service] f:reason: {} [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:updatedReplicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] args: [e2e-llm-inference-service] - hf://facebook/opt-125m [e2e-llm-inference-service] - /mnt/models [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] - name: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] - name: S3_USE_HTTPS [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: S3_ENDPOINT [e2e-llm-inference-service] value: s3-service.kserve:8333 [e2e-llm-inference-service] - name: AWS_ENDPOINT_URL [e2e-llm-inference-service] value: http://s3-service.kserve:8333 [e2e-llm-inference-service] - name: S3_VERIFY_SSL [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: AWS_CA_BUNDLE [e2e-llm-inference-service] value: /etc/ssl/custom-certs/cabundle.crt [e2e-llm-inference-service] - name: AWS_CA_BUNDLE_CONFIGMAP [e2e-llm-inference-service] value: odh-kserve-custom-ca-bundle [e2e-llm-inference-service] - name: HF_HUB_ENABLE_HF_TRANSFER [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_HIGH_PERFORMANCE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_NUM_CONCURRENT_RANGE_GETS [e2e-llm-inference-service] value: '8' [e2e-llm-inference-service] - name: STORAGE_ALLOW_PATTERNS [e2e-llm-inference-service] value: '["tokenizer.json", "tokenizer_config.json", "special_tokens_map.json", [e2e-llm-inference-service] "vocab.json", "merges.txt", "config.json", "generation_config.json"]' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - auth-enabled-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: single-profile-handler\n- type: queue-scorer\n- type:\ [e2e-llm-inference-service] \ kv-cache-utilization-scorer\n- type: prefix-cache-scorer\n- type: no-hit-lru-scorer\n\ [e2e-llm-inference-service] - type: max-score-picker\nschedulingProfiles:\n- name: default\n plugins:\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: kv-cache-utilization-scorer\n\ [e2e-llm-inference-service] \ weight: 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef:\ [e2e-llm-inference-service] \ no-hit-lru-scorer\n weight: 2\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '6' [e2e-llm-inference-service] memory: 16Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: auth-enabled-test-epp-sa [e2e-llm-inference-service] serviceAccount: auth-enabled-test-epp-sa [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] strategy: [e2e-llm-inference-service] type: Recreate [e2e-llm-inference-service] revisionHistoryLimit: 10 [e2e-llm-inference-service] progressDeadlineSeconds: 600 [e2e-llm-inference-service] status: [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] updatedReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] - type: Available [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] reason: MinimumReplicasAvailable [e2e-llm-inference-service] message: Deployment has minimum availability. [e2e-llm-inference-service] - type: Progressing [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] lastUpdateTime: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] lastTransitionTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] reason: NewReplicaSetAvailable [e2e-llm-inference-service] message: ReplicaSet "auth-enabled-test-kserve-router-scheduler-794498b458" has [e2e-llm-inference-service] successfully progressed. [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-6c997987dd [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 17474aac-8c70-466d-bfc0-ffdde13897f3 [e2e-llm-inference-service] resourceVersion: '28784' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] pod-template-hash: 6c997987dd [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '2' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: auth-enabled-test-kserve [e2e-llm-inference-service] uid: 6d81965c-bb64-4052-8e35-8d58e0e45bf4 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"6d81965c-bb64-4052-8e35-8d58e0e45bf4"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:llm-d.ai/role: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_CACHE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HOME"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"TORCHINDUCTOR_CACHE_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"USER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_CPU_KVCACHE_SPACE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_ENABLE_V1_MULTIPROCESSING"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"VLLM_LOGGING_LEVEL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":8000,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:startupProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:httpGet: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:path: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:scheme: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/dev/shm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] k:{"mountPath":"/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"storage-initializer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"AWS_ACCESS_KEY_ID"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE_CONFIGMAP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_ENDPOINT_URL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_SECRET_ACCESS_KEY"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_ENABLE_HF_TRANSFER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_HIGH_PERFORMANCE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_NUM_CONCURRENT_RANGE_GETS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_ENDPOINT"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_USE_HTTPS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_VERIFY_SSL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"TOKIO_WORKER_THREADS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"dshm"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:medium: {} [e2e-llm-inference-service] f:sizeLimit: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"home"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"kserve-provision-location"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"model-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tmp-dir"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] pod-template-hash: 6c997987dd [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] pod-template-hash: 6c997987dd [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] emptyDir: [e2e-llm-inference-service] medium: Memory [e2e-llm-inference-service] sizeLimit: 1Gi [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] args: [e2e-llm-inference-service] - hf://facebook/opt-125m [e2e-llm-inference-service] - /mnt/models [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: TOKIO_WORKER_THREADS [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] - name: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] - name: S3_USE_HTTPS [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: S3_ENDPOINT [e2e-llm-inference-service] value: s3-service.kserve:8333 [e2e-llm-inference-service] - name: AWS_ENDPOINT_URL [e2e-llm-inference-service] value: http://s3-service.kserve:8333 [e2e-llm-inference-service] - name: S3_VERIFY_SSL [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: AWS_CA_BUNDLE [e2e-llm-inference-service] value: /etc/ssl/custom-certs/cabundle.crt [e2e-llm-inference-service] - name: AWS_CA_BUNDLE_CONFIGMAP [e2e-llm-inference-service] value: odh-kserve-custom-ca-bundle [e2e-llm-inference-service] - name: HF_HUB_ENABLE_HF_TRANSFER [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_HIGH_PERFORMANCE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_NUM_CONCURRENT_RANGE_GETS [e2e-llm-inference-service] value: '8' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/bash [e2e-llm-inference-service] - -c [e2e-llm-inference-service] - "if [ -f /etc/profile.d/ibm-aiu-setup.sh ]; then\n source /etc/profile.d/ibm-aiu-setup.sh\n\ [e2e-llm-inference-service] fi\n\nif [ \"$KSERVE_INFER_ROCE\" = \"true\" ]; then\n echo \"Trying to\ [e2e-llm-inference-service] \ infer RoCE configs ... \"\n grep -H . /sys/class/infiniband/*/ports/*/gids/*\ [e2e-llm-inference-service] \ 2>/dev/null\n grep -H . /sys/class/infiniband/*/ports/*/gid_attrs/types/*\ [e2e-llm-inference-service] \ 2>/dev/null\n\n cat /proc/driver/nvidia/params\n\n KSERVE_INFER_IB_GID_INDEX_GREP=${KSERVE_INFER_IB_GID_INDEX_GREP:-\"\ [e2e-llm-inference-service] RoCE v2\"}\n\n echo \"[Infer RoCE] Discovering active HCAs ...\"\n active_hcas=()\n\ [e2e-llm-inference-service] \ # Loop through all mlx5 devices found in sysfs\n for hca_dir in /sys/class/infiniband/mlx5_*;\ [e2e-llm-inference-service] \ do\n # Ensure it's a directory before proceeding\n if [ -d \"\ [e2e-llm-inference-service] $hca_dir\" ]; then\n hca_name=$(basename \"$hca_dir\")\n \ [e2e-llm-inference-service] \ port_state_file=\"$hca_dir/ports/1/state\" # Assume port 1\n \ [e2e-llm-inference-service] \ type_file=\"$hca_dir/ports/1/gid_attrs/types/*\"\n\n echo\ [e2e-llm-inference-service] \ \"[Infer RoCE] Check if the port state file ${port_state_file} exists\ [e2e-llm-inference-service] \ and contains 'ACTIVE'\"\n if [ -f \"$port_state_file\" ] && grep\ [e2e-llm-inference-service] \ -q \"ACTIVE\" \"$port_state_file\" && grep -q \"${KSERVE_INFER_IB_GID_INDEX_GREP}\"\ [e2e-llm-inference-service] \ ${type_file} 2>/dev/null; then\n echo \"[Infer RoCE] Found\ [e2e-llm-inference-service] \ active HCA: $hca_name\"\n active_hcas+=(\"$hca_name\")\n\ [e2e-llm-inference-service] \ else\n echo \"[Infer RoCE] Skipping inactive or\ [e2e-llm-inference-service] \ down HCA: $hca_name\"\n fi\n fi\n done\n\n # Check if\ [e2e-llm-inference-service] \ we found any active HCAs\n if [ ${#active_hcas[@]} -gt 0 ]; then\n \ [e2e-llm-inference-service] \ # Join the array elements with a comma\n hca_port_pairs=()\n \ [e2e-llm-inference-service] \ for hca in \"${active_hcas[@]}\"; do\n hca_port_pairs+=(\"\ [e2e-llm-inference-service] ${hca}:1\")\n done\n\n active_hca_list=$(IFS=,; echo \"${active_hcas[*]}\"\ [e2e-llm-inference-service] )\n hca_port_pairs_list=$(IFS=,; echo \"${hca_port_pairs[*]}\")\n \ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Setting active HCAs: ${active_hca_list}\"\n \ [e2e-llm-inference-service] \ export NCCL_IB_HCA=${NCCL_IB_HCA:-${active_hca_list}}\n export\ [e2e-llm-inference-service] \ NVSHMEM_HCA_LIST=${NVSHMEM_HCA_LIST:-${hca_port_pairs_list}}\n export\ [e2e-llm-inference-service] \ UCX_NET_DEVICES=${UCX_NET_DEVICES:-${hca_port_pairs_list}}\n\n echo\ [e2e-llm-inference-service] \ \"[Infer RoCE] NCCL_IB_HCA=${NCCL_IB_HCA}\"\n echo \"[Infer RoCE]\ [e2e-llm-inference-service] \ NVSHMEM_HCA_LIST=${NVSHMEM_HCA_LIST}\"\n echo \"[Infer RoCE] UCX_NET_DEVICES=${UCX_NET_DEVICES}\"\ [e2e-llm-inference-service] \n else\n echo \"[Infer RoCE] WARNING: No active RoCE HCAs found.\ [e2e-llm-inference-service] \ NCCL_IB_HCA will not be set.\"\n fi\n\n if [ ${#active_hcas[@]} -gt\ [e2e-llm-inference-service] \ 0 ]; then\n echo \"[Infer RoCE] Finding GID_INDEX for each active\ [e2e-llm-inference-service] \ HCA (SR-IOV compatible)...\"\n\n # For SR-IOV environments, find\ [e2e-llm-inference-service] \ the most common IPv4 RoCE v2 GID index across all HCAs\n declare\ [e2e-llm-inference-service] \ -A gid_index_count\n declare -A hca_gid_index\n\n for hca_name\ [e2e-llm-inference-service] \ in \"${active_hcas[@]}\"; do\n echo \"[Infer RoCE] Processing\ [e2e-llm-inference-service] \ HCA: ${hca_name}\"\n\n # Find all RoCE v2 IPv4 GIDs for this\ [e2e-llm-inference-service] \ HCA and count by index\n for tpath in /sys/class/infiniband/${hca_name}/ports/1/gid_attrs/types/*;\ [e2e-llm-inference-service] \ do\n if grep -q \"${KSERVE_INFER_IB_GID_INDEX_GREP}\" \"\ [e2e-llm-inference-service] $tpath\" 2>/dev/null; then\n idx=$(basename \"$tpath\"\ [e2e-llm-inference-service] )\n gid_file=\"/sys/class/infiniband/${hca_name}/ports/1/gids/${idx}\"\ [e2e-llm-inference-service] \n # Check for IPv4 GID (contains ffff:)\n \ [e2e-llm-inference-service] \ if [ -f \"$gid_file\" ] && grep -q \"ffff:\" \"$gid_file\"; then\n\ [e2e-llm-inference-service] \ gid_value=$(cat \"$gid_file\" 2>/dev/null || echo\ [e2e-llm-inference-service] \ \"\")\n echo \"[Infer RoCE] Found IPv4 RoCE v2 GID\ [e2e-llm-inference-service] \ for ${hca_name}: index=${idx}, gid=${gid_value}\"\n \ [e2e-llm-inference-service] \ hca_gid_index[\"${hca_name}\"]=\"${idx}\"\n gid_index_count[\"\ [e2e-llm-inference-service] ${idx}\"]=$((${gid_index_count[\"${idx}\"]} + 1))\n \ [e2e-llm-inference-service] \ break # Use first found IPv4 GID per HCA\n fi\n \ [e2e-llm-inference-service] \ fi\n done\n done\n\n # Find the most common\ [e2e-llm-inference-service] \ GID index (most likely to be consistent across nodes)\n best_gid_index=\"\ [e2e-llm-inference-service] \"\n max_count=0\n for idx in \"${!gid_index_count[@]}\"; do\n\ [e2e-llm-inference-service] \ count=${gid_index_count[\"${idx}\"]}\n echo \"[Infer\ [e2e-llm-inference-service] \ RoCE] GID_INDEX ${idx} found on ${count} HCAs\"\n if [ $count\ [e2e-llm-inference-service] \ -gt $max_count ]; then\n max_count=$count\n \ [e2e-llm-inference-service] \ best_gid_index=\"$idx\"\n fi\n done\n\n # Use deterministic\ [e2e-llm-inference-service] \ fallback if tied - prefer index 3 (SR-IOV standard)\n if [ ${#gid_index_count[@]}\ [e2e-llm-inference-service] \ -gt 1 ]; then\n echo \"[Infer RoCE] Multiple GID indices found,\ [e2e-llm-inference-service] \ selecting most common: ${best_gid_index}\"\n # If there's a tie,\ [e2e-llm-inference-service] \ prefer index 3 as it's most common in SR-IOV setups\n if [ -n\ [e2e-llm-inference-service] \ \"${gid_index_count['3']}\" ] && [ \"${gid_index_count['3']}\" -eq \"\ [e2e-llm-inference-service] $max_count\" ]; then\n best_gid_index=\"3\"\n \ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Using deterministic fallback: GID_INDEX=3 (SR-IOV\ [e2e-llm-inference-service] \ standard)\"\n fi\n fi\n\n # Check if GID_INDEX is already\ [e2e-llm-inference-service] \ set via environment variables\n if [ -n \"${NCCL_IB_GID_INDEX}\"\ [e2e-llm-inference-service] \ ]; then\n echo \"[Infer RoCE] Using pre-configured NCCL_IB_GID_INDEX=${NCCL_IB_GID_INDEX}\ [e2e-llm-inference-service] \ from environment\"\n export NVSHMEM_IB_GID_INDEX=${NVSHMEM_IB_GID_INDEX:-$NCCL_IB_GID_INDEX}\n\ [e2e-llm-inference-service] \ export UCX_IB_GID_INDEX=${UCX_IB_GID_INDEX:-$NCCL_IB_GID_INDEX}\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Using pre-configured GID_INDEX=${NCCL_IB_GID_INDEX}\ [e2e-llm-inference-service] \ for NCCL, NVSHMEM, and UCX\"\n elif [ -n \"$best_gid_index\" ]; then\n\ [e2e-llm-inference-service] \ echo \"[Infer RoCE] Selected GID_INDEX: ${best_gid_index} (found\ [e2e-llm-inference-service] \ on ${max_count} HCAs)\"\n\n export NCCL_IB_GID_INDEX=${NCCL_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \ export NVSHMEM_IB_GID_INDEX=${NVSHMEM_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \ export UCX_IB_GID_INDEX=${UCX_IB_GID_INDEX:-$best_gid_index}\n\ [e2e-llm-inference-service] \n echo \"[Infer RoCE] Exported GID_INDEX=${best_gid_index} for\ [e2e-llm-inference-service] \ NCCL, NVSHMEM, and UCX\"\n else\n echo \"[Infer RoCE] ERROR:\ [e2e-llm-inference-service] \ No valid IPv4 ${KSERVE_INFER_IB_GID_INDEX_GREP} GID_INDEX found on any\ [e2e-llm-inference-service] \ HCA.\"\n fi\n else\n echo \"[Infer RoCE] No active HCAs found,\ [e2e-llm-inference-service] \ skipping GID_INDEX inference.\"\n fi\nfi\n\n# --disable-access-log-for-endpoints\ [e2e-llm-inference-service] \ landed in vLLM 0.16.0 (vllm-project/vllm#30011).\n# Older versions still\ [e2e-llm-inference-service] \ need the blanket --disable-uvicorn-access-log.\nACCESS_LOG_ARGS=\"--disable-uvicorn-access-log\"\ [e2e-llm-inference-service] \nVLLM_VERSION=$(vllm --version 2>/dev/null | tail -1 | awk '{print $NF}')\n\ [e2e-llm-inference-service] echo \"[access-log-detect] vllm version='${VLLM_VERSION}'\"\nif [[ \"$VLLM_VERSION\"\ [e2e-llm-inference-service] \ =~ ^[0-9]+\\.[0-9]+ ]] && [ \"$(printf '%s\\n%s\\n' \"0.16.0\" \"${VLLM_VERSION}\"\ [e2e-llm-inference-service] \ | sort -V | head -1)\" = \"0.16.0\" ]; then\n ACCESS_LOG_ARGS=\"--disable-access-log-for-endpoints\ [e2e-llm-inference-service] \ /health,/metrics,/ping\"\nfi\necho \"[access-log-detect] selected ACCESS_LOG_ARGS='${ACCESS_LOG_ARGS}'\"\ [e2e-llm-inference-service] \n\n# --shutdown-timeout landed in vLLM 0.18.0 (vllm-project/vllm#36666).\n\ [e2e-llm-inference-service] SHUTDOWN_TIMEOUT_ARGS=\"\"\nif [[ \"$VLLM_VERSION\" =~ ^[0-9]+\\.[0-9]+\ [e2e-llm-inference-service] \ ]] && [ \"$(printf '%s\\n%s\\n' \"0.18.0\" \"${VLLM_VERSION}\" | sort\ [e2e-llm-inference-service] \ -V | head -1)\" = \"0.18.0\" ]; then\n SHUTDOWN_TIMEOUT_ARGS=\"--shutdown-timeout\ [e2e-llm-inference-service] \ 40\"\nfi\n\n# --kv-transfer-config with OffloadingConnector requires vLLM\ [e2e-llm-inference-service] \ 0.22.0+ (vllm-project/vllm#40020).\nKV_TRANSFER_ARGS=\"\"\nif [[ \"$VLLM_VERSION\"\ [e2e-llm-inference-service] \ =~ ^[0-9]+\\.[0-9]+ ]] && [ \"$(printf '%s\\n%s\\n' \"0.22.0\" \"${VLLM_VERSION}\"\ [e2e-llm-inference-service] \ | sort -V | head -1)\" = \"0.22.0\" ]; then\n if [[ \"${VLLM_ADDITIONAL_ARGS:-}\"\ [e2e-llm-inference-service] \ != *\"--kv-transfer-config\"* ]] && [[ \"${VLLM_ADDITIONAL_ARGS:-}\" !=\ [e2e-llm-inference-service] \ *\"--kv_transfer_config\"* ]] && [[ \"$*\" != *\"--kv-transfer-config\"\ [e2e-llm-inference-service] * ]] && [[ \"$*\" != *\"--kv_transfer_config\"* ]]; then\n KV_TRANSFER_ARGS=\"\ [e2e-llm-inference-service] \"\n fi\nfi\n\neval \"exec vllm serve /mnt/models \\\n --served-model-name\ [e2e-llm-inference-service] \ \"facebook/opt-125m\" \"publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m\"\ [e2e-llm-inference-service] \ \\\n --port 8000 \\\n ${ACCESS_LOG_ARGS} \\\n ${SHUTDOWN_TIMEOUT_ARGS}\ [e2e-llm-inference-service] \ \\\n ${KV_TRANSFER_ARGS} \\\n \\\n --enable-ssl-refresh \\\n --ssl-certfile\ [e2e-llm-inference-service] \ /var/run/kserve/tls/tls.crt \\\n --ssl-keyfile /var/run/kserve/tls/tls.key\ [e2e-llm-inference-service] \ \\\n ${VLLM_ADDITIONAL_ARGS} \\\n $@\"" [e2e-llm-inference-service] - -- [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - containerPort: 8000 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: HOME [e2e-llm-inference-service] value: /home [e2e-llm-inference-service] - name: VLLM_LOGGING_LEVEL [e2e-llm-inference-service] value: DEBUG [e2e-llm-inference-service] - name: VLLM_CPU_KVCACHE_SPACE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: VLLM_ENABLE_V1_MULTIPROCESSING [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: USER [e2e-llm-inference-service] value: nonroot [e2e-llm-inference-service] - name: TORCHINDUCTOR_CACHE_DIR [e2e-llm-inference-service] value: /tmp/torchinductor-cache [e2e-llm-inference-service] - name: HF_HUB_CACHE [e2e-llm-inference-service] value: /models [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '2' [e2e-llm-inference-service] memory: 7Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 200m [e2e-llm-inference-service] memory: 2Gi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: home [e2e-llm-inference-service] mountPath: /home [e2e-llm-inference-service] - name: tmp-dir [e2e-llm-inference-service] mountPath: /tmp [e2e-llm-inference-service] - name: dshm [e2e-llm-inference-service] mountPath: /dev/shm [e2e-llm-inference-service] - name: model-cache [e2e-llm-inference-service] mountPath: /models [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 10 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 1 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 2 [e2e-llm-inference-service] startupProbe: [e2e-llm-inference-service] httpGet: [e2e-llm-inference-service] path: /health [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] scheme: HTTPS [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 60 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: false [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler-794498b458 [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 7794b4d5-fa45-48ea-877e-68aef52cbdec [e2e-llm-inference-service] resourceVersion: '27161' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 794498b458 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] deployment.kubernetes.io/desired-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/max-replicas: '1' [e2e-llm-inference-service] deployment.kubernetes.io/revision: '1' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: apps/v1 [e2e-llm-inference-service] kind: Deployment [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler [e2e-llm-inference-service] uid: 8c613b3a-ecbb-4e4d-8a9a-2633ca7781a3 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/desired-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/max-replicas: {} [e2e-llm-inference-service] f:deployment.kubernetes.io/revision: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"8c613b3a-ecbb-4e4d-8a9a-2633ca7781a3"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] f:selector: {} [e2e-llm-inference-service] f:template: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/version: {} [e2e-llm-inference-service] f:certificates.kserve.io/expiration-v2: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:pod-template-hash: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] f:containers: [e2e-llm-inference-service] k:{"name":"main"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"SSL_CERT_DIR"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:lifecycle: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:preStop: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:command: {} [e2e-llm-inference-service] f:livenessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:ports: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"containerPort":5557,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9002,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9003,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] k:{"containerPort":9090,"protocol":"TCP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:containerPort: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:protocol: {} [e2e-llm-inference-service] f:readinessProbe: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureThreshold: {} [e2e-llm-inference-service] f:grpc: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:port: {} [e2e-llm-inference-service] f:service: {} [e2e-llm-inference-service] f:initialDelaySeconds: {} [e2e-llm-inference-service] f:periodSeconds: {} [e2e-llm-inference-service] f:successThreshold: {} [e2e-llm-inference-service] f:timeoutSeconds: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:securityContext: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:allowPrivilegeEscalation: {} [e2e-llm-inference-service] f:capabilities: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:drop: {} [e2e-llm-inference-service] f:readOnlyRootFilesystem: {} [e2e-llm-inference-service] f:runAsNonRoot: {} [e2e-llm-inference-service] f:seccompProfile: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:type: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/tmp/tokenizer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"mountPath":"/var/run/kserve/tls"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:readOnly: {} [e2e-llm-inference-service] f:dnsPolicy: {} [e2e-llm-inference-service] f:initContainers: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"storage-initializer"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:args: {} [e2e-llm-inference-service] f:env: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"AWS_ACCESS_KEY_ID"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_CA_BUNDLE_CONFIGMAP"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_ENDPOINT_URL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"AWS_SECRET_ACCESS_KEY"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:valueFrom: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:secretKeyRef: {} [e2e-llm-inference-service] k:{"name":"HF_HUB_ENABLE_HF_TRANSFER"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_HIGH_PERFORMANCE"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"HF_XET_NUM_CONCURRENT_RANGE_GETS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_ENDPOINT"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_USE_HTTPS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"S3_VERIFY_SSL"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] k:{"name":"STORAGE_ALLOW_PATTERNS"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:value: {} [e2e-llm-inference-service] f:image: {} [e2e-llm-inference-service] f:imagePullPolicy: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:resources: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:limits: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:requests: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:cpu: {} [e2e-llm-inference-service] f:memory: {} [e2e-llm-inference-service] f:terminationMessagePath: {} [e2e-llm-inference-service] f:terminationMessagePolicy: {} [e2e-llm-inference-service] f:volumeMounts: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"mountPath":"/mnt/models"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:mountPath: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:restartPolicy: {} [e2e-llm-inference-service] f:schedulerName: {} [e2e-llm-inference-service] f:securityContext: {} [e2e-llm-inference-service] f:serviceAccount: {} [e2e-llm-inference-service] f:serviceAccountName: {} [e2e-llm-inference-service] f:terminationGracePeriodSeconds: {} [e2e-llm-inference-service] f:volumes: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"name":"kserve-provision-location"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tls-certs"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:secret: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:defaultMode: {} [e2e-llm-inference-service] f:secretName: {} [e2e-llm-inference-service] k:{"name":"tokenizer-cache"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-tmp"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] k:{"name":"tokenizer-uds"}: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:emptyDir: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:availableReplicas: {} [e2e-llm-inference-service] f:fullyLabeledReplicas: {} [e2e-llm-inference-service] f:observedGeneration: {} [e2e-llm-inference-service] f:readyReplicas: {} [e2e-llm-inference-service] f:replicas: {} [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] spec: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 794498b458 [e2e-llm-inference-service] template: [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 794498b458 [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] app.kubernetes.io/version: 0.9.0 [e2e-llm-inference-service] certificates.kserve.io/expiration-v2: 'true' [e2e-llm-inference-service] spec: [e2e-llm-inference-service] volumes: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] secret: [e2e-llm-inference-service] secretName: auth-enabled-test-kserve-self-signed-certs [e2e-llm-inference-service] defaultMode: 420 [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-tmp [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: tokenizer-cache [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] emptyDir: {} [e2e-llm-inference-service] initContainers: [e2e-llm-inference-service] - name: storage-initializer [e2e-llm-inference-service] image: quay.io/opendatahub/kserve-storage-initializer@sha256:038cd29f3e031cdfd5fa54ba896bbecc69cd255eda6b470c2f7bce5282357bb8 [e2e-llm-inference-service] args: [e2e-llm-inference-service] - hf://facebook/opt-125m [e2e-llm-inference-service] - /mnt/models [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_ACCESS_KEY_ID [e2e-llm-inference-service] - name: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] valueFrom: [e2e-llm-inference-service] secretKeyRef: [e2e-llm-inference-service] name: seaweedfs-s3-creds [e2e-llm-inference-service] key: AWS_SECRET_ACCESS_KEY [e2e-llm-inference-service] - name: S3_USE_HTTPS [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: S3_ENDPOINT [e2e-llm-inference-service] value: s3-service.kserve:8333 [e2e-llm-inference-service] - name: AWS_ENDPOINT_URL [e2e-llm-inference-service] value: http://s3-service.kserve:8333 [e2e-llm-inference-service] - name: S3_VERIFY_SSL [e2e-llm-inference-service] value: '0' [e2e-llm-inference-service] - name: AWS_CA_BUNDLE [e2e-llm-inference-service] value: /etc/ssl/custom-certs/cabundle.crt [e2e-llm-inference-service] - name: AWS_CA_BUNDLE_CONFIGMAP [e2e-llm-inference-service] value: odh-kserve-custom-ca-bundle [e2e-llm-inference-service] - name: HF_HUB_ENABLE_HF_TRANSFER [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_HIGH_PERFORMANCE [e2e-llm-inference-service] value: '1' [e2e-llm-inference-service] - name: HF_XET_NUM_CONCURRENT_RANGE_GETS [e2e-llm-inference-service] value: '8' [e2e-llm-inference-service] - name: STORAGE_ALLOW_PATTERNS [e2e-llm-inference-service] value: '["tokenizer.json", "tokenizer_config.json", "special_tokens_map.json", [e2e-llm-inference-service] "vocab.json", "merges.txt", "config.json", "generation_config.json"]' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '1' [e2e-llm-inference-service] memory: 24Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 100m [e2e-llm-inference-service] memory: 100Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: kserve-provision-location [e2e-llm-inference-service] mountPath: /mnt/models [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] image: quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0 [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /app/epp [e2e-llm-inference-service] - --pool-name [e2e-llm-inference-service] - auth-enabled-test-inference-pool [e2e-llm-inference-service] - --pool-namespace [e2e-llm-inference-service] - e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] - --zap-encoder [e2e-llm-inference-service] - json [e2e-llm-inference-service] - --grpc-port [e2e-llm-inference-service] - '9002' [e2e-llm-inference-service] - --grpc-health-port [e2e-llm-inference-service] - '9003' [e2e-llm-inference-service] - --enable-cert-reload=true [e2e-llm-inference-service] - --secure-serving=true [e2e-llm-inference-service] - --model-server-metrics-scheme=https [e2e-llm-inference-service] - --cert-path=/var/run/kserve/tls [e2e-llm-inference-service] args: [e2e-llm-inference-service] - --config-text [e2e-llm-inference-service] - "apiVersion: inference.networking.x-k8s.io/v1alpha1\nkind: EndpointPickerConfig\n\ [e2e-llm-inference-service] plugins:\n- type: single-profile-handler\n- type: queue-scorer\n- type:\ [e2e-llm-inference-service] \ kv-cache-utilization-scorer\n- type: prefix-cache-scorer\n- type: no-hit-lru-scorer\n\ [e2e-llm-inference-service] - type: max-score-picker\nschedulingProfiles:\n- name: default\n plugins:\n\ [e2e-llm-inference-service] \ - pluginRef: queue-scorer\n weight: 2\n - pluginRef: kv-cache-utilization-scorer\n\ [e2e-llm-inference-service] \ weight: 2\n - pluginRef: prefix-cache-scorer\n weight: 3\n - pluginRef:\ [e2e-llm-inference-service] \ no-hit-lru-scorer\n weight: 2\n - pluginRef: max-score-picker\n" [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] containerPort: 9002 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] containerPort: 9003 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] containerPort: 9090 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] containerPort: 5557 [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] env: [e2e-llm-inference-service] - name: SSL_CERT_DIR [e2e-llm-inference-service] value: /var/run/kserve/tls:/var/run/secrets/kubernetes.io/serviceaccount:/etc/pki/tls/certs [e2e-llm-inference-service] resources: [e2e-llm-inference-service] limits: [e2e-llm-inference-service] cpu: '6' [e2e-llm-inference-service] memory: 16Gi [e2e-llm-inference-service] requests: [e2e-llm-inference-service] cpu: 256m [e2e-llm-inference-service] memory: 500Mi [e2e-llm-inference-service] volumeMounts: [e2e-llm-inference-service] - name: tls-certs [e2e-llm-inference-service] readOnly: true [e2e-llm-inference-service] mountPath: /var/run/kserve/tls [e2e-llm-inference-service] - name: tokenizer-uds [e2e-llm-inference-service] mountPath: /tmp/tokenizer [e2e-llm-inference-service] livenessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: liveness [e2e-llm-inference-service] initialDelaySeconds: 5 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] readinessProbe: [e2e-llm-inference-service] grpc: [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] service: readiness [e2e-llm-inference-service] initialDelaySeconds: 30 [e2e-llm-inference-service] timeoutSeconds: 1 [e2e-llm-inference-service] periodSeconds: 10 [e2e-llm-inference-service] successThreshold: 1 [e2e-llm-inference-service] failureThreshold: 3 [e2e-llm-inference-service] lifecycle: [e2e-llm-inference-service] preStop: [e2e-llm-inference-service] exec: [e2e-llm-inference-service] command: [e2e-llm-inference-service] - /bin/sleep [e2e-llm-inference-service] - '15' [e2e-llm-inference-service] terminationMessagePath: /dev/termination-log [e2e-llm-inference-service] terminationMessagePolicy: FallbackToLogsOnError [e2e-llm-inference-service] imagePullPolicy: IfNotPresent [e2e-llm-inference-service] securityContext: [e2e-llm-inference-service] capabilities: [e2e-llm-inference-service] drop: [e2e-llm-inference-service] - ALL [e2e-llm-inference-service] runAsNonRoot: true [e2e-llm-inference-service] readOnlyRootFilesystem: true [e2e-llm-inference-service] allowPrivilegeEscalation: false [e2e-llm-inference-service] seccompProfile: [e2e-llm-inference-service] type: RuntimeDefault [e2e-llm-inference-service] restartPolicy: Always [e2e-llm-inference-service] terminationGracePeriodSeconds: 60 [e2e-llm-inference-service] dnsPolicy: ClusterFirst [e2e-llm-inference-service] serviceAccountName: auth-enabled-test-epp-sa [e2e-llm-inference-service] serviceAccount: auth-enabled-test-epp-sa [e2e-llm-inference-service] securityContext: {} [e2e-llm-inference-service] schedulerName: default-scheduler [e2e-llm-inference-service] status: [e2e-llm-inference-service] replicas: 1 [e2e-llm-inference-service] fullyLabeledReplicas: 1 [e2e-llm-inference-service] readyReplicas: 1 [e2e-llm-inference-service] availableReplicas: 1 [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] apiVersion: apps/v1 [e2e-llm-inference-service] kind: ReplicaSet [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-rb [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: c1fdd115-a0e0-4aa0-b67e-925684cf2c49 [e2e-llm-inference-service] resourceVersion: '26161' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] name: auth-enabled-test-epp-sa [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] apiGroup: rbac.authorization.k8s.io [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] name: auth-enabled-test-epp-role [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-role [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 17081a28-7a24-4ffd-845c-44b7be8510a3 [e2e-llm-inference-service] resourceVersion: '26159' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - inferencemodels [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - inferencepoolimports [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - llm-d.ai [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - discovery.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - endpointslices [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] - create [e2e-llm-inference-service] - update [e2e-llm-inference-service] - patch [e2e-llm-inference-service] - delete [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - coordination.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - leases [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-service-ml27j [e2e-llm-inference-service] generateName: auth-enabled-test-epp-service- [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 2e5433d6-b15f-4247-9ee2-e92e099e41fd [e2e-llm-inference-service] resourceVersion: '27160' [e2e-llm-inference-service] generation: 3 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io [e2e-llm-inference-service] kubernetes.io/service-name: auth-enabled-test-epp-service [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-epp-service [e2e-llm-inference-service] uid: 4c53d392-887a-412f-9eef-69b74e71136c [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:37Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:addressType: {} [e2e-llm-inference-service] f:endpoints: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpointslice.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:kubernetes.io/service-name: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"4c53d392-887a-412f-9eef-69b74e71136c"}: {} [e2e-llm-inference-service] f:ports: {} [e2e-llm-inference-service] addressType: IPv4 [e2e-llm-inference-service] endpoints: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.134.0.35 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler-794498b458-mt7dc [e2e-llm-inference-service] uid: b99ce9e9-0067-46b0-a74e-9b05501a3c02 [e2e-llm-inference-service] nodeName: ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: grpc [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9002 [e2e-llm-inference-service] - name: grpc-health [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9003 [e2e-llm-inference-service] - name: metrics [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 9090 [e2e-llm-inference-service] - name: zmq [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 5557 [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] kind: EndpointSlice [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc-9lf9x [e2e-llm-inference-service] generateName: auth-enabled-test-kserve-workload-svc- [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 89775787-ac2f-4289-afb7-e6d195c2136f [e2e-llm-inference-service] resourceVersion: '28781' [e2e-llm-inference-service] generation: 3 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] endpointslice.kubernetes.io/managed-by: endpointslice-controller.k8s.io [e2e-llm-inference-service] kubernetes.io/service-name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] endpoints.kubernetes.io/last-change-trigger-time: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: v1 [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] uid: 0cb27ec7-b426-439d-93b3-877bb0444800 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: kube-controller-manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-20T10:16:33Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:addressType: {} [e2e-llm-inference-service] f:endpoints: {} [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:endpoints.kubernetes.io/last-change-trigger-time: {} [e2e-llm-inference-service] f:generateName: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:endpointslice.kubernetes.io/managed-by: {} [e2e-llm-inference-service] f:kubernetes.io/service-name: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"0cb27ec7-b426-439d-93b3-877bb0444800"}: {} [e2e-llm-inference-service] f:ports: {} [e2e-llm-inference-service] addressType: IPv4 [e2e-llm-inference-service] endpoints: [e2e-llm-inference-service] - addresses: [e2e-llm-inference-service] - 10.134.0.34 [e2e-llm-inference-service] conditions: [e2e-llm-inference-service] ready: true [e2e-llm-inference-service] serving: true [e2e-llm-inference-service] terminating: false [e2e-llm-inference-service] targetRef: [e2e-llm-inference-service] kind: Pod [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] name: auth-enabled-test-kserve-6c997987dd-r59r7 [e2e-llm-inference-service] uid: 55e3988c-bb7d-4c82-bb5d-f8bb06dcd862 [e2e-llm-inference-service] nodeName: ip-10-0-138-196.ec2.internal [e2e-llm-inference-service] zone: us-east-1a [e2e-llm-inference-service] ports: [e2e-llm-inference-service] - name: https [e2e-llm-inference-service] protocol: TCP [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] appProtocol: https [e2e-llm-inference-service] apiVersion: discovery.k8s.io/v1 [e2e-llm-inference-service] kind: EndpointSlice [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-rb [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: c1fdd115-a0e0-4aa0-b67e-925684cf2c49 [e2e-llm-inference-service] resourceVersion: '26161' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:roleRef: {} [e2e-llm-inference-service] f:subjects: {} [e2e-llm-inference-service] userNames: [e2e-llm-inference-service] - system:serviceaccount:e2e-test-llm-auth-enabled-requires-token-d4ed2c08:auth-enabled-test-epp-sa [e2e-llm-inference-service] groupNames: null [e2e-llm-inference-service] subjects: [e2e-llm-inference-service] - kind: ServiceAccount [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] name: auth-enabled-test-epp-sa [e2e-llm-inference-service] roleRef: [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] name: auth-enabled-test-epp-role [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: RoleBinding [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-epp-role [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] uid: 17081a28-7a24-4ffd-845c-44b7be8510a3 [e2e-llm-inference-service] resourceVersion: '26159' [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] apiVersion: rbac.authorization.k8s.io/v1 [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:rules: {} [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - '' [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - pods [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.k8s.io [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodels [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - inferencepools [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - inference.networking.x-k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - inferencepoolimports [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - llm-d.ai [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - inferencemodelrewrites [e2e-llm-inference-service] - inferenceobjectives [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - discovery.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - endpointslices [e2e-llm-inference-service] - verbs: [e2e-llm-inference-service] - create [e2e-llm-inference-service] - delete [e2e-llm-inference-service] - get [e2e-llm-inference-service] - list [e2e-llm-inference-service] - patch [e2e-llm-inference-service] - update [e2e-llm-inference-service] - watch [e2e-llm-inference-service] attributeRestrictions: null [e2e-llm-inference-service] apiGroups: [e2e-llm-inference-service] - coordination.k8s.io [e2e-llm-inference-service] resources: [e2e-llm-inference-service] - leases [e2e-llm-inference-service] apiVersion: authorization.openshift.io/v1 [e2e-llm-inference-service] kind: Role [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/inference-pool-migrated: v1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] generation: 2 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/inference-pool-migrated: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-20T10:14:18Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-route [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26614' [e2e-llm-inference-service] uid: 3c8b9b26-bf98-48fe-aa7a-1ed9bd86cd7f [e2e-llm-inference-service] spec: [e2e-llm-inference-service] parentRefs: [e2e-llm-inference-service] - group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions/ [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions/ [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses/ [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: / [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] message: Route was valid [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] message: All references resolved [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] controllerName: openshift.io/gateway-controller/v1 [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] message: Object affected by AuthPolicy [openshift-ingress/openshift-ai-inference-authn] [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: kuadrant.io/AuthPolicyAffected [e2e-llm-inference-service] controllerName: kuadrant.io/policy-controller [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] kind: HTTPRoute [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] annotations: [e2e-llm-inference-service] serving.kserve.io/inference-pool-migrated: v1 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] generation: 2 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:annotations: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:serving.kserve.io/inference-pool-migrated: {} [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1beta1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] - apiVersion: gateway.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-20T10:14:18Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-route [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26614' [e2e-llm-inference-service] uid: 3c8b9b26-bf98-48fe-aa7a-1ed9bd86cd7f [e2e-llm-inference-service] spec: [e2e-llm-inference-service] parentRefs: [e2e-llm-inference-service] - group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] rules: [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/completions/ [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/chat/completions/ [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/responses/ [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: /v1/messages/ [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/chat/completions [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/chat/completions [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/responses [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/responses [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: inference.networking.k8s.io [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: /v1/messages [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m/v1/messages [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] filters: [e2e-llm-inference-service] - type: URLRewrite [e2e-llm-inference-service] urlRewrite: [e2e-llm-inference-service] path: [e2e-llm-inference-service] replacePrefixMatch: / [e2e-llm-inference-service] type: ReplacePrefixMatch [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: /e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] - backendRefs: [e2e-llm-inference-service] - group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] port: 8000 [e2e-llm-inference-service] weight: 1 [e2e-llm-inference-service] matches: [e2e-llm-inference-service] - headers: [e2e-llm-inference-service] - name: X-Gateway-Model-Name [e2e-llm-inference-service] type: Exact [e2e-llm-inference-service] value: publishers/e2e-test-llm-auth-enabled-requires-token-d4ed2c08/models/facebook/opt-125m [e2e-llm-inference-service] path: [e2e-llm-inference-service] type: PathPrefix [e2e-llm-inference-service] value: / [e2e-llm-inference-service] timeouts: [e2e-llm-inference-service] backendRequest: 0s [e2e-llm-inference-service] request: 0s [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] message: Route was valid [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] message: All references resolved [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] controllerName: openshift.io/gateway-controller/v1 [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] message: Object affected by AuthPolicy [openshift-ingress/openshift-ai-inference-authn] [e2e-llm-inference-service] observedGeneration: 2 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: kuadrant.io/AuthPolicyAffected [e2e-llm-inference-service] controllerName: kuadrant.io/policy-controller [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:appProtocol: {} [e2e-llm-inference-service] f:endpointPickerRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureMode: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:port: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:number: {} [e2e-llm-inference-service] f:selector: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:matchLabels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:targetPorts: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] - apiVersion: inference.networking.k8s.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:status: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:parents: {} [e2e-llm-inference-service] manager: pilot-discovery [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] subresource: status [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26600' [e2e-llm-inference-service] uid: b984ee8f-7699-429a-81ad-f77afa64a0b3 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] appProtocol: http [e2e-llm-inference-service] endpointPickerRef: [e2e-llm-inference-service] failureMode: FailOpen [e2e-llm-inference-service] group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-epp-service [e2e-llm-inference-service] port: [e2e-llm-inference-service] number: 9002 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] matchLabels: [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] targetPorts: [e2e-llm-inference-service] - number: 8000 [e2e-llm-inference-service] status: [e2e-llm-inference-service] parents: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] message: Referenced by an HTTPRoute accepted by the parentRef Gateway [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] reason: Accepted [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] - lastTransitionTime: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] message: Referenced ExtensionRef resolved successfully [e2e-llm-inference-service] observedGeneration: 1 [e2e-llm-inference-service] reason: ResolvedRefs [e2e-llm-inference-service] status: 'True' [e2e-llm-inference-service] type: ResolvedRefs [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: networking.istio.io [e2e-llm-inference-service] kind: Gateway [e2e-llm-inference-service] name: openshift-ai-inference [e2e-llm-inference-service] namespace: openshift-ingress [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-scheduler [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26200' [e2e-llm-inference-service] uid: 22425b46-8265-4d83-b26c-a321986e9117 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-epp-service.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-epp-service.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26605' [e2e-llm-inference-service] uid: 937dc711-39e3-497d-a1be-58d03bfd9cff [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-inference-pool-ip-0e6361b2.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:caCertificates: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26229' [e2e-llm-inference-service] uid: af0c44dc-7737-4976-8dce-d9f4b2e459c4 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] caCertificates: /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt [e2e-llm-inference-service] insecureSkipVerify: false [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-scheduler [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26200' [e2e-llm-inference-service] uid: 22425b46-8265-4d83-b26c-a321986e9117 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-epp-service.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-epp-service.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26605' [e2e-llm-inference-service] uid: 937dc711-39e3-497d-a1be-58d03bfd9cff [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-inference-pool-ip-0e6361b2.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1beta1 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:caCertificates: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26229' [e2e-llm-inference-service] uid: af0c44dc-7737-4976-8dce-d9f4b2e459c4 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] caCertificates: /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt [e2e-llm-inference-service] insecureSkipVerify: false [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-scheduler [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26200' [e2e-llm-inference-service] uid: 22425b46-8265-4d83-b26c-a321986e9117 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-epp-service.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-epp-service.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-shadow-service [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:17Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-shadow-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26605' [e2e-llm-inference-service] uid: 937dc711-39e3-497d-a1be-58d03bfd9cff [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-inference-pool-ip-0e6361b2.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] insecureSkipVerify: true [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: networking.istio.io/v1alpha3 [e2e-llm-inference-service] kind: DestinationRule [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] llm-d.ai/managed: 'true' [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: networking.istio.io/v1 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:llm-d.ai/managed: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:exportTo: {} [e2e-llm-inference-service] f:host: {} [e2e-llm-inference-service] f:trafficPolicy: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:tls: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:caCertificates: {} [e2e-llm-inference-service] f:insecureSkipVerify: {} [e2e-llm-inference-service] f:mode: {} [e2e-llm-inference-service] f:sni: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:04Z' [e2e-llm-inference-service] name: auth-enabled-test-kserve-workload-svc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26229' [e2e-llm-inference-service] uid: af0c44dc-7737-4976-8dce-d9f4b2e459c4 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] exportTo: [e2e-llm-inference-service] - '*' [e2e-llm-inference-service] host: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] trafficPolicy: [e2e-llm-inference-service] tls: [e2e-llm-inference-service] caCertificates: /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt [e2e-llm-inference-service] insecureSkipVerify: false [e2e-llm-inference-service] mode: SIMPLE [e2e-llm-inference-service] sni: auth-enabled-test-kserve-workload-svc.e2e-test-llm-auth-enabled-requires-token-d4ed2c08.svc.cluster.local [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 apiVersion: inference.networking.x-k8s.io/v1alpha2 [e2e-llm-inference-service] kind: InferencePool [e2e-llm-inference-service] metadata: [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] generation: 1 [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] managedFields: [e2e-llm-inference-service] - apiVersion: inference.networking.x-k8s.io/v1alpha2 [e2e-llm-inference-service] fieldsType: FieldsV1 [e2e-llm-inference-service] fieldsV1: [e2e-llm-inference-service] f:metadata: [e2e-llm-inference-service] f:labels: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/component: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:ownerReferences: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] k:{"uid":"d6930790-b1d8-403d-b9f9-dc77e30178e5"}: {} [e2e-llm-inference-service] f:spec: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:extensionRef: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:failureMode: {} [e2e-llm-inference-service] f:group: {} [e2e-llm-inference-service] f:kind: {} [e2e-llm-inference-service] f:name: {} [e2e-llm-inference-service] f:portNumber: {} [e2e-llm-inference-service] f:selector: [e2e-llm-inference-service] .: {} [e2e-llm-inference-service] f:app.kubernetes.io/name: {} [e2e-llm-inference-service] f:app.kubernetes.io/part-of: {} [e2e-llm-inference-service] f:kserve.io/component: {} [e2e-llm-inference-service] f:targetPortNumber: {} [e2e-llm-inference-service] manager: manager [e2e-llm-inference-service] operation: Update [e2e-llm-inference-service] time: '2026-07-20T10:14:03Z' [e2e-llm-inference-service] name: auth-enabled-test-inference-pool [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] ownerReferences: [e2e-llm-inference-service] - apiVersion: serving.kserve.io/v1alpha2 [e2e-llm-inference-service] blockOwnerDeletion: true [e2e-llm-inference-service] controller: true [e2e-llm-inference-service] kind: LLMInferenceService [e2e-llm-inference-service] name: auth-enabled-test [e2e-llm-inference-service] uid: d6930790-b1d8-403d-b9f9-dc77e30178e5 [e2e-llm-inference-service] resourceVersion: '26180' [e2e-llm-inference-service] uid: 0c3ab655-d063-4682-bd07-07bf61f3afe4 [e2e-llm-inference-service] spec: [e2e-llm-inference-service] extensionRef: [e2e-llm-inference-service] failureMode: FailOpen [e2e-llm-inference-service] group: '' [e2e-llm-inference-service] kind: Service [e2e-llm-inference-service] name: auth-enabled-test-epp-service [e2e-llm-inference-service] portNumber: 9002 [e2e-llm-inference-service] selector: [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] targetPortNumber: 8000 [e2e-llm-inference-service] status: [e2e-llm-inference-service] parent: [e2e-llm-inference-service] - conditions: [e2e-llm-inference-service] - lastTransitionTime: '1970-01-01T00:00:00Z' [e2e-llm-inference-service] message: Waiting for controller [e2e-llm-inference-service] reason: Pending [e2e-llm-inference-service] status: Unknown [e2e-llm-inference-service] type: Accepted [e2e-llm-inference-service] parentRef: [e2e-llm-inference-service] group: gateway.networking.k8s.io [e2e-llm-inference-service] kind: Status [e2e-llm-inference-service] name: default [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-6c997987dd-r59r7 [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:18:37Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-workload [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] kserve.io/component: workload [e2e-llm-inference-service] llm-d.ai/role: both [e2e-llm-inference-service] pod-template-hash: 6c997987dd [e2e-llm-inference-service] timestamp: '2026-07-20T10:18:30Z' [e2e-llm-inference-service] window: 18.401s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 103674419n [e2e-llm-inference-service] memory: 2410920Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:264 --- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:diagnostic.py:265 metadata: [e2e-llm-inference-service] name: auth-enabled-test-kserve-router-scheduler-794498b458-mt7dc [e2e-llm-inference-service] namespace: e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] creationTimestamp: '2026-07-20T10:18:37Z' [e2e-llm-inference-service] labels: [e2e-llm-inference-service] app.kubernetes.io/component: llminferenceservice-router-scheduler [e2e-llm-inference-service] app.kubernetes.io/name: auth-enabled-test [e2e-llm-inference-service] app.kubernetes.io/part-of: llminferenceservice [e2e-llm-inference-service] pod-template-hash: 794498b458 [e2e-llm-inference-service] timestamp: '2026-07-20T10:18:30Z' [e2e-llm-inference-service] window: 18.321s [e2e-llm-inference-service] containers: [e2e-llm-inference-service] - name: main [e2e-llm-inference-service] usage: [e2e-llm-inference-service] cpu: 54948419n [e2e-llm-inference-service] memory: 33104Ki [e2e-llm-inference-service] apiVersion: metrics.k8s.io/v1beta1 [e2e-llm-inference-service] kind: PodMetrics [e2e-llm-inference-service] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:205 ✅ Deleted RoleBinding auth-enabled-test-test-sa-binding [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:211 ✅ Deleted Role auth-enabled-test-test-sa-role [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:217 ✅ Deleted ServiceAccount auth-enabled-test-test-sa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_auth.py:379 ⏭️ Skipping deletion of auth-enabled-test due to test failure (SKIP_DELETION_ON_FAILURE=True) [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_auth_enabled_requires_token] [2026-07-20T10:18:38.076811] end - ❌ 279.457s: Expected 200 with token, got 403: [e2e-llm-inference-service] assert 403 == 200 [e2e-llm-inference-service] + where 403 = .status_code [e2e-llm-inference-service] ---------------------------- Captured log teardown ----------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:97 Deleted test namespace e2e-test-llm-auth-enabled-requires-token-d4ed2c08 [e2e-llm-inference-service] _ test_llm_autoscaling_hpa_deployment[router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-hpa] _ [e2e-llm-inference-service] [gw0] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-no-replicas', 'prometheus-scrape', 'scaling-hpa'], prom... {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.autoscaling_hpa [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator-no-replicas", [e2e-llm-inference-service] "prometheus-scrape", [e2e-llm-inference-service] "scaling-hpa", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="autoscale-hpa-deploy", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_autoscaling_hpa_deployment(test_case: TestCase): [e2e-llm-inference-service] """HPA + Deployment: VA and HPA exist; pods scale up under load.""" [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] kserve_client = _new_kserve_client() [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] ns = test_case.namespace [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > _create_and_wait(kserve_client, test_case) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:540: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-no-replicas', 'prometheus-scrape', 'scaling-hpa'], prom... {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] def _create_and_wait(kserve_client, test_case): [e2e-llm-inference-service] """Create LLMISVC and wait for it to be ready.""" [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] > wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:480: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kin...e-hpa-4c186bcf'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None}, 900) [e2e-llm-inference-service] kwargs = {}, func_name = 'wait_for_llm_isvc_ready' [e2e-llm-inference-service] timestamp_start = '2026-07-20T10:24:15.869788', start_time = 1784543055.87021 [e2e-llm-inference-service] duration = 900.8121633529663, timestamp_end = '2026-07-20T10:39:16.682376' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] given = {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] ...toscale-hpa-4c186bcf'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None} [e2e-llm-inference-service] timeout_seconds = 900 [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] given: V1alpha1LLMInferenceService, [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] return True [e2e-llm-inference-service] [e2e-llm-inference-service] > return wait_for(assert_llm_isvc_ready, timeout=timeout_seconds, interval=1.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1255: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .assert_llm_isvc_ready at 0x7f8d21052700> [e2e-llm-inference-service] timeout = 900, interval = 1.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] last_msg = None [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1266: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1250: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:74 Created test namespace e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret seaweedfs-s3-creds from kserve-ci-e2e-test to e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret storage-config from kserve-ci-e2e-test to e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:170 Patched default SA in e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 with secret seaweedfs-s3-creds [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:154 ConfigMap odh-kserve-custom-ca-bundle already exists in e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:151 Copied ConfigMap odh-trusted-ca-bundle from kserve-ci-e2e-test to e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig router-managed-autoscale-hpa-de-ec1dce8b in namespace e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig router-managed-autoscale-hpa-de-ec1dce8b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig router-managed-autoscale-hpa-de-ec1dce8b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig workload-llmd-simulator-no-repl-38916baa in namespace e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig workload-llmd-simulator-no-repl-38916baa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig workload-llmd-simulator-no-repl-38916baa [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig prometheus-scrape-autoscale-hpa-4c186bcf in namespace e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig prometheus-scrape-autoscale-hpa-4c186bcf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig prometheus-scrape-autoscale-hpa-4c186bcf [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig scaling-hpa-autoscale-hpa-deplo-347a3180 in namespace e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig scaling-hpa-autoscale-hpa-deplo-347a3180 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig scaling-hpa-autoscale-hpa-deplo-347a3180 [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_autoscaling_hpa_deployment] [2026-07-20T10:24:15.805102] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'workload-llmd-simulator-no-replicas', 'prometheus-scrape', 'scaling-hpa'], prompt='KServe is a', service_name='autoscale-hpa-deploy', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, namespace='e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4', before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-de-ec1dce8b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-38916baa'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-4c186bcf'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-07-20T10:24:15.817987] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-de-ec1dce8b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-38916baa'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-4c186bcf'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-07-20T10:24:15.869602] end - ✅ in 0.051s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-07-20T10:24:15.869788] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-de-ec1dce8b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-38916baa'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-4c186bcf'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 3: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 4: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 5: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:1270 Timed out waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_llm_isvc_ready] [2026-07-20T10:39:16.682376] end - ❌ 900.812s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [delete_llmisvc] [2026-07-20T10:39:16.682592] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-de-ec1dce8b'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-38916baa'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-4c186bcf'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-deplo-347a3180'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: 1 pod(s) for autoscale-hpa-deploy still terminating: ['autoscale-hpa-deploy-kserve-66746bd8b9-747xw'] [e2e-llm-inference-service] assert not ['autoscale-hpa-deploy-kserve-66746bd8b9-747xw'] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [delete_llmisvc] [2026-07-20T10:39:21.758698] end - ✅ in 5.076s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_autoscaling_hpa_deployment] [2026-07-20T10:39:21.758863] end - ❌ 905.953s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:24:21Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ---------------------------- Captured log teardown ----------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:97 Deleted test namespace e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4 [e2e-llm-inference-service] _ test_llm_autoscaling_keda_deployment[router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-keda] _ [e2e-llm-inference-service] [gw0] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-no-replicas', 'prometheus-scrape', 'scaling-keda'], pro... {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.autoscaling_keda [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator-no-replicas", [e2e-llm-inference-service] "prometheus-scrape", [e2e-llm-inference-service] "scaling-keda", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="autoscale-keda-deploy", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_single_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_autoscaling_keda_deployment(test_case: TestCase): [e2e-llm-inference-service] """KEDA + Deployment: VA and ScaledObject exist; no HPA; pods scale up under load.""" [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] kserve_client = _new_kserve_client() [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] ns = test_case.namespace [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > _create_and_wait(kserve_client, test_case) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:604: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-no-replicas', 'prometheus-scrape', 'scaling-keda'], pro... {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] def _create_and_wait(kserve_client, test_case): [e2e-llm-inference-service] """Create LLMISVC and wait for it to be ready.""" [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] > wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:480: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kin...e-ked-101f2a9d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None}, 900) [e2e-llm-inference-service] kwargs = {}, func_name = 'wait_for_llm_isvc_ready' [e2e-llm-inference-service] timestamp_start = '2026-07-20T10:39:22.870344', start_time = 1784543962.8706155 [e2e-llm-inference-service] duration = 900.0523600578308, timestamp_end = '2026-07-20T10:54:22.922983' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] given = {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] ...toscale-ked-101f2a9d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None} [e2e-llm-inference-service] timeout_seconds = 900 [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] given: V1alpha1LLMInferenceService, [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] return True [e2e-llm-inference-service] [e2e-llm-inference-service] > return wait_for(assert_llm_isvc_ready, timeout=timeout_seconds, interval=1.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1255: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .assert_llm_isvc_ready at 0x7f8d20616a20> [e2e-llm-inference-service] timeout = 900, interval = 1.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] last_msg = None [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1266: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1250: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:74 Created test namespace e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret seaweedfs-s3-creds from kserve-ci-e2e-test to e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret storage-config from kserve-ci-e2e-test to e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:170 Patched default SA in e2e-test-llm-autoscaling-keda-deployment-b2150d0b with secret seaweedfs-s3-creds [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:154 ConfigMap odh-kserve-custom-ca-bundle already exists in e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:151 Copied ConfigMap odh-trusted-ca-bundle from kserve-ci-e2e-test to e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig router-managed-autoscale-keda-d-27e06c40 in namespace e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig router-managed-autoscale-keda-d-27e06c40 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig router-managed-autoscale-keda-d-27e06c40 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig workload-llmd-simulator-no-repl-da49e827 in namespace e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig workload-llmd-simulator-no-repl-da49e827 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig workload-llmd-simulator-no-repl-da49e827 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig prometheus-scrape-autoscale-ked-101f2a9d in namespace e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig prometheus-scrape-autoscale-ked-101f2a9d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig prometheus-scrape-autoscale-ked-101f2a9d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig scaling-keda-autoscale-keda-dep-1ac84077 in namespace e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig scaling-keda-autoscale-keda-dep-1ac84077 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig scaling-keda-autoscale-keda-dep-1ac84077 [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_autoscaling_keda_deployment] [2026-07-20T10:39:22.612433] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'workload-llmd-simulator-no-replicas', 'prometheus-scrape', 'scaling-keda'], prompt='KServe is a', service_name='autoscale-keda-deploy', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, namespace='e2e-test-llm-autoscaling-keda-deployment-b2150d0b', before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-deployment-b2150d0b', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-d-27e06c40'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-da49e827'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-101f2a9d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-07-20T10:39:22.629631] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-deployment-b2150d0b', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-d-27e06c40'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-da49e827'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-101f2a9d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-07-20T10:39:22.870212] end - ✅ in 0.240s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-07-20T10:39:22.870344] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-deployment-b2150d0b', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-d-27e06c40'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-da49e827'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-101f2a9d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 6: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 7: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 8: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:1270 Timed out waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_llm_isvc_ready] [2026-07-20T10:54:22.922983] end - ❌ 900.052s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [delete_llmisvc] [2026-07-20T10:54:22.923107] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-deploy', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-deployment-b2150d0b', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-d-27e06c40'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-no-repl-da49e827'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-101f2a9d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-dep-1ac84077'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: 1 pod(s) for autoscale-keda-deploy still terminating: ['autoscale-keda-deploy-kserve-77fc6d7cf9-4trw8'] [e2e-llm-inference-service] assert not ['autoscale-keda-deploy-kserve-77fc6d7cf9-4trw8'] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [delete_llmisvc] [2026-07-20T10:54:28.030027] end - ✅ in 5.107s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_autoscaling_keda_deployment] [2026-07-20T10:54:28.030147] end - ❌ 905.417s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:39:29Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ---------------------------- Captured log teardown ----------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:97 Deleted test namespace e2e-test-llm-autoscaling-keda-deployment-b2150d0b [e2e-llm-inference-service] _ test_llm_autoscaling_hpa_lws[router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-hpa] _ [e2e-llm-inference-service] [gw0] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-lws', 'prometheus-scrape', 'scaling-hpa'], prompt='KSer... {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.autoscaling_hpa [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator-lws", [e2e-llm-inference-service] "prometheus-scrape", [e2e-llm-inference-service] "scaling-hpa", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="autoscale-hpa-lws", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_multi_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_autoscaling_hpa_lws(test_case: TestCase): [e2e-llm-inference-service] """HPA + LWS: VA and HPA exist; pods scale under load.""" [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] kserve_client = _new_kserve_client() [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] ns = test_case.namespace [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > _create_and_wait(kserve_client, test_case) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:668: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-lws', 'prometheus-scrape', 'scaling-hpa'], prompt='KSer... {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] def _create_and_wait(kserve_client, test_case): [e2e-llm-inference-service] """Create LLMISVC and wait for it to be ready.""" [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] > wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:480: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kin...ale-hpa-b29acdba'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None}, 900) [e2e-llm-inference-service] kwargs = {}, func_name = 'wait_for_llm_isvc_ready' [e2e-llm-inference-service] timestamp_start = '2026-07-20T10:54:28.467008', start_time = 1784544868.4673026 [e2e-llm-inference-service] duration = 900.705540895462, timestamp_end = '2026-07-20T11:09:29.172846' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] given = {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] ...autoscale-hpa-b29acdba'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None} [e2e-llm-inference-service] timeout_seconds = 900 [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] given: V1alpha1LLMInferenceService, [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] return True [e2e-llm-inference-service] [e2e-llm-inference-service] > return wait_for(assert_llm_isvc_ready, timeout=timeout_seconds, interval=1.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1255: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .assert_llm_isvc_ready at 0x7f8d206177e0> [e2e-llm-inference-service] timeout = 900, interval = 1.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] last_msg = None [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1266: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:54:44Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1250: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:74 Created test namespace e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret seaweedfs-s3-creds from kserve-ci-e2e-test to e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret storage-config from kserve-ci-e2e-test to e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:170 Patched default SA in e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 with secret seaweedfs-s3-creds [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:154 ConfigMap odh-kserve-custom-ca-bundle already exists in e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:151 Copied ConfigMap odh-trusted-ca-bundle from kserve-ci-e2e-test to e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig router-managed-autoscale-hpa-lw-1aa98714 in namespace e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig router-managed-autoscale-hpa-lw-1aa98714 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig router-managed-autoscale-hpa-lw-1aa98714 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig workload-llmd-simulator-lws-aut-fe7a55cc in namespace e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig workload-llmd-simulator-lws-aut-fe7a55cc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig workload-llmd-simulator-lws-aut-fe7a55cc [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig prometheus-scrape-autoscale-hpa-b29acdba in namespace e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig prometheus-scrape-autoscale-hpa-b29acdba [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig prometheus-scrape-autoscale-hpa-b29acdba [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig scaling-hpa-autoscale-hpa-lws-b344a3ff in namespace e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig scaling-hpa-autoscale-hpa-lws-b344a3ff [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig scaling-hpa-autoscale-hpa-lws-b344a3ff [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_autoscaling_hpa_lws] [2026-07-20T10:54:28.390487] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'workload-llmd-simulator-lws', 'prometheus-scrape', 'scaling-hpa'], prompt='KServe is a', service_name='autoscale-hpa-lws', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, namespace='e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2', before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-lw-1aa98714'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-fe7a55cc'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-b29acdba'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-07-20T10:54:28.402971] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-lw-1aa98714'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-fe7a55cc'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-b29acdba'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-07-20T10:54:28.466867] end - ✅ in 0.064s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-07-20T10:54:28.467008] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-lw-1aa98714'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-fe7a55cc'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-b29acdba'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'LWS is progressing', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:54:44Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 9: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 10: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 11: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:1270 Timed out waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:54:44Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_llm_isvc_ready] [2026-07-20T11:09:29.172846] end - ❌ 900.706s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:54:44Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [delete_llmisvc] [2026-07-20T11:09:29.172924] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-hpa-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-hpa-lw-1aa98714'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-fe7a55cc'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-hpa-b29acdba'}, [e2e-llm-inference-service] {'name': 'scaling-hpa-autoscale-hpa-lws-b344a3ff'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: 2 pod(s) for autoscale-hpa-lws still terminating: ['autoscale-hpa-lws-kserve-mn-0', 'autoscale-hpa-lws-kserve-mn-0-1'] [e2e-llm-inference-service] assert not ['autoscale-hpa-lws-kserve-mn-0', 'autoscale-hpa-lws-kserve-mn-0-1'] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [delete_llmisvc] [2026-07-20T11:09:34.252077] end - ✅ in 5.079s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_autoscaling_hpa_lws] [2026-07-20T11:09:34.252247] end - ❌ 905.861s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T10:54:44Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T10:54:33Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ---------------------------- Captured log teardown ----------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:97 Deleted test namespace e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2 [e2e-llm-inference-service] _ test_llm_autoscaling_keda_lws[router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-keda] _ [e2e-llm-inference-service] [gw0] linux -- Python 3.11.13 /workspace/source/python/kserve/.venv/bin/python [e2e-llm-inference-service] [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-lws', 'prometheus-scrape', 'scaling-keda'], prompt='KSe... {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] @pytest.mark.autoscaling_keda [e2e-llm-inference-service] @pytest.mark.parametrize( [e2e-llm-inference-service] "test_case", [e2e-llm-inference-service] [ [e2e-llm-inference-service] pytest.param( [e2e-llm-inference-service] TestCase( [e2e-llm-inference-service] base_refs=[ [e2e-llm-inference-service] "router-managed", [e2e-llm-inference-service] "workload-llmd-simulator-lws", [e2e-llm-inference-service] "prometheus-scrape", [e2e-llm-inference-service] "scaling-keda", [e2e-llm-inference-service] ], [e2e-llm-inference-service] prompt="KServe is a", [e2e-llm-inference-service] service_name="autoscale-keda-lws", [e2e-llm-inference-service] ), [e2e-llm-inference-service] marks=[ [e2e-llm-inference-service] pytest.mark.cluster_cpu, [e2e-llm-inference-service] pytest.mark.cluster_multi_node, [e2e-llm-inference-service] pytest.mark.llmd_simulator, [e2e-llm-inference-service] ], [e2e-llm-inference-service] ), [e2e-llm-inference-service] ], [e2e-llm-inference-service] indirect=["test_case"], [e2e-llm-inference-service] ids=generate_test_id, [e2e-llm-inference-service] ) [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def test_llm_autoscaling_keda_lws(test_case: TestCase): [e2e-llm-inference-service] """KEDA + LWS: VA and ScaledObject exist; pods scale under load.""" [e2e-llm-inference-service] inject_k8s_proxy() [e2e-llm-inference-service] kserve_client = _new_kserve_client() [e2e-llm-inference-service] service_name = test_case.llm_service.metadata.name [e2e-llm-inference-service] ns = test_case.namespace [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > _create_and_wait(kserve_client, test_case) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:726: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] test_case = TestCase(base_refs=['router-managed', 'workload-llmd-simulator-lws', 'prometheus-scrape', 'scaling-keda'], prompt='KSe... {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m') [e2e-llm-inference-service] [e2e-llm-inference-service] def _create_and_wait(kserve_client, test_case): [e2e-llm-inference-service] """Create LLMISVC and wait for it to be ready.""" [e2e-llm-inference-service] create_llmisvc(kserve_client, test_case.llm_service) [e2e-llm-inference-service] > wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client, test_case.llm_service, test_case.wait_timeout [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_autoscaling_wva.py:480: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] args = (, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kin...e-ked-231d315d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None}, 900) [e2e-llm-inference-service] kwargs = {}, func_name = 'wait_for_llm_isvc_ready' [e2e-llm-inference-service] timestamp_start = '2026-07-20T11:09:35.035005', start_time = 1784545775.0353525 [e2e-llm-inference-service] duration = 900.581330537796, timestamp_end = '2026-07-20T11:24:35.616685' [e2e-llm-inference-service] [e2e-llm-inference-service] @functools.wraps(func) [e2e-llm-inference-service] def wrapper(*args, **kwargs): [e2e-llm-inference-service] func_name = func.__name__ [e2e-llm-inference-service] [e2e-llm-inference-service] timestamp_start = datetime.now().isoformat() [e2e-llm-inference-service] logger.info( [e2e-llm-inference-service] f"[{func_name}] [{timestamp_start}] start - args={args}, kwargs={kwargs}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] start_time = time.time() [e2e-llm-inference-service] [e2e-llm-inference-service] try: [e2e-llm-inference-service] > result = func(*args, **kwargs) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/logging.py:40: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] kserve_client = [e2e-llm-inference-service] given = {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] ...toscale-ked-231d315d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None} [e2e-llm-inference-service] timeout_seconds = 900 [e2e-llm-inference-service] [e2e-llm-inference-service] @log_execution [e2e-llm-inference-service] def wait_for_llm_isvc_ready( [e2e-llm-inference-service] kserve_client: KServeClient, [e2e-llm-inference-service] given: V1alpha1LLMInferenceService, [e2e-llm-inference-service] timeout_seconds: int = 900, [e2e-llm-inference-service] ) -> str: [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] return True [e2e-llm-inference-service] [e2e-llm-inference-service] > return wait_for(assert_llm_isvc_ready, timeout=timeout_seconds, interval=1.0) [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1255: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] assertion_fn = .assert_llm_isvc_ready at 0x7f8d20616ca0> [e2e-llm-inference-service] timeout = 900, interval = 1.0 [e2e-llm-inference-service] [e2e-llm-inference-service] def wait_for( [e2e-llm-inference-service] assertion_fn: Callable[[], Any], timeout: float = 5.0, interval: float = 0.1 [e2e-llm-inference-service] ) -> Any: [e2e-llm-inference-service] """Wait for the assertion to succeed within timeout.""" [e2e-llm-inference-service] deadline = time.time() + timeout [e2e-llm-inference-service] last_msg = None [e2e-llm-inference-service] while True: [e2e-llm-inference-service] try: [e2e-llm-inference-service] > return assertion_fn() [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1266: [e2e-llm-inference-service] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [e2e-llm-inference-service] [e2e-llm-inference-service] def assert_llm_isvc_ready(): [e2e-llm-inference-service] out = get_llmisvc( [e2e-llm-inference-service] kserve_client, [e2e-llm-inference-service] given.metadata.name, [e2e-llm-inference-service] given.metadata.namespace, [e2e-llm-inference-service] given.api_version.split("/")[1], [e2e-llm-inference-service] ) [e2e-llm-inference-service] [e2e-llm-inference-service] if "status" not in out: [e2e-llm-inference-service] raise AssertionError("No status found in LLM inference service") [e2e-llm-inference-service] [e2e-llm-inference-service] status = out["status"] [e2e-llm-inference-service] if "conditions" not in status: [e2e-llm-inference-service] raise AssertionError("No conditions found in status") [e2e-llm-inference-service] [e2e-llm-inference-service] expected_true_conditions = {"Ready", "WorkloadsReady", "RouterReady"} [e2e-llm-inference-service] got_true_conditions = set() [e2e-llm-inference-service] [e2e-llm-inference-service] conditions = status["conditions"] [e2e-llm-inference-service] [e2e-llm-inference-service] for condition in conditions: [e2e-llm-inference-service] if condition.get("status") == "True": [e2e-llm-inference-service] got_true_conditions.add(condition.get("type")) [e2e-llm-inference-service] [e2e-llm-inference-service] missing_conditions = expected_true_conditions - got_true_conditions [e2e-llm-inference-service] if missing_conditions: [e2e-llm-inference-service] > raise AssertionError( [e2e-llm-inference-service] f"Missing true conditions: {missing_conditions}, expected {expected_true_conditions}, got {conditions}" [e2e-llm-inference-service] ) [e2e-llm-inference-service] E AssertionError: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T11:09:53Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:1250: AssertionError [e2e-llm-inference-service] ------------------------------ Captured log setup ------------------------------ [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:74 Created test namespace e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret seaweedfs-s3-creds from kserve-ci-e2e-test to e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:123 Copied secret storage-config from kserve-ci-e2e-test to e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:170 Patched default SA in e2e-test-llm-autoscaling-keda-lws-e541a132 with secret seaweedfs-s3-creds [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:154 ConfigMap odh-kserve-custom-ca-bundle already exists in e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:151 Copied ConfigMap odh-trusted-ca-bundle from kserve-ci-e2e-test to e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig router-managed-autoscale-keda-l-78828c4a in namespace e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig router-managed-autoscale-keda-l-78828c4a [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig router-managed-autoscale-keda-l-78828c4a [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig workload-llmd-simulator-lws-aut-1696d0b7 in namespace e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig workload-llmd-simulator-lws-aut-1696d0b7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig workload-llmd-simulator-lws-aut-1696d0b7 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig prometheus-scrape-autoscale-ked-231d315d in namespace e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig prometheus-scrape-autoscale-ked-231d315d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig prometheus-scrape-autoscale-ked-231d315d [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1639 Checking LLMInferenceServiceConfig scaling-keda-autoscale-keda-lws-1337f511 in namespace e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1665 Resource not found, creating LLMInferenceServiceConfig scaling-keda-autoscale-keda-lws-1337f511 [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1675 ✓ Successfully created LLMInferenceServiceConfig scaling-keda-autoscale-keda-lws-1337f511 [e2e-llm-inference-service] ------------------------------ Captured log call ------------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [test_llm_autoscaling_keda_lws] [2026-07-20T11:09:34.868869] start - args=(), kwargs={'test_case': TestCase(base_refs=['router-managed', 'workload-llmd-simulator-lws', 'prometheus-scrape', 'scaling-keda'], prompt='KServe is a', service_name='autoscale-keda-lws', endpoint='/v1/completions', max_tokens=20, payload_formatter=None, response_assertion=, wait_timeout=900, response_timeout=60, extra_headers=None, url_getter=None, expected_gateway=None, namespace='e2e-test-llm-autoscaling-keda-lws-e541a132', before_test=[], after_test=[], peers=[], llm_service={'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-lws-e541a132', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-l-78828c4a'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-1696d0b7'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-231d315d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None}, model_name='facebook/opt-125m')} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:fixtures.py:1690 No HTTP proxy configured for k8s client [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [create_llmisvc] [2026-07-20T11:09:34.881695] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-lws-e541a132', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-l-78828c4a'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-1696d0b7'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-231d315d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [create_llmisvc] [2026-07-20T11:09:35.034860] end - ✅ in 0.153s [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [wait_for_llm_isvc_ready] [2026-07-20T11:09:35.035005] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-lws-e541a132', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-l-78828c4a'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-1696d0b7'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-231d315d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None}, 900), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: No conditions found in status [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'LWS is progressing', 'reason': 'Progressing', 'severity': 'Info', 'status': 'False', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T11:09:53Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 12: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 13: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] INFO common.gateway_proxy_istio:gateway_proxy_istio.py:421 Snapshot 14: restarts={"kserve-ci-e2e-test/router-gateway-1-openshift-default": "error", "kserve-ci-e2e-test/router-gateway-2-openshift-default": "error", "openshift-ingress/openshift-ai-inference-openshift-default": "1\t{\"running\":{\"startedAt\":\"2026-07-20T10:11:21Z\"}}"} resources={"httproutes": 0, "llminferenceservices": 0, "authpolicies": 0} [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:test_llm_inference_service.py:1270 Timed out waiting: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T11:09:53Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [wait_for_llm_isvc_ready] [2026-07-20T11:24:35.616685] end - ❌ 900.581s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T11:09:53Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:34 [delete_llmisvc] [2026-07-20T11:24:35.616776] start - args=(, {'api_version': 'serving.kserve.io/v1alpha1', [e2e-llm-inference-service] 'kind': 'LLMInferenceService', [e2e-llm-inference-service] 'metadata': {'annotations': None, [e2e-llm-inference-service] 'creation_timestamp': None, [e2e-llm-inference-service] 'deletion_grace_period_seconds': None, [e2e-llm-inference-service] 'deletion_timestamp': None, [e2e-llm-inference-service] 'finalizers': None, [e2e-llm-inference-service] 'generate_name': None, [e2e-llm-inference-service] 'generation': None, [e2e-llm-inference-service] 'labels': None, [e2e-llm-inference-service] 'managed_fields': None, [e2e-llm-inference-service] 'name': 'autoscale-keda-lws', [e2e-llm-inference-service] 'namespace': 'e2e-test-llm-autoscaling-keda-lws-e541a132', [e2e-llm-inference-service] 'owner_references': None, [e2e-llm-inference-service] 'resource_version': None, [e2e-llm-inference-service] 'self_link': None, [e2e-llm-inference-service] 'uid': None}, [e2e-llm-inference-service] 'spec': {'baseRefs': [{'name': 'router-managed-autoscale-keda-l-78828c4a'}, [e2e-llm-inference-service] {'name': 'workload-llmd-simulator-lws-aut-1696d0b7'}, [e2e-llm-inference-service] {'name': 'prometheus-scrape-autoscale-ked-231d315d'}, [e2e-llm-inference-service] {'name': 'scaling-keda-autoscale-keda-lws-1337f511'}]}, [e2e-llm-inference-service] 'status': None}), kwargs={} [e2e-llm-inference-service] INFO e2e.llmisvc.logging:test_llm_inference_service.py:1273 Waiting: 2 pod(s) for autoscale-keda-lws still terminating: ['autoscale-keda-lws-kserve-mn-0', 'autoscale-keda-lws-kserve-mn-0-1'] [e2e-llm-inference-service] assert not ['autoscale-keda-lws-kserve-mn-0', 'autoscale-keda-lws-kserve-mn-0-1'] [e2e-llm-inference-service] INFO e2e.llmisvc.logging:logging.py:43 [delete_llmisvc] [2026-07-20T11:24:40.685665] end - ✅ in 5.069s [e2e-llm-inference-service] ERROR e2e.llmisvc.logging:logging.py:48 [test_llm_autoscaling_keda_lws] [2026-07-20T11:24:40.685787] end - ❌ 905.817s: Missing true conditions: {'WorkloadsReady', 'RouterReady', 'Ready'}, expected {'WorkloadsReady', 'RouterReady', 'Ready'}, got [{'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'severity': 'Info', 'status': 'False', 'type': 'MainWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'severity': 'Info', 'status': 'True', 'type': 'PresetsCombined'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'Ready'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'status': 'Unknown', 'type': 'RouterReady'}, {'lastTransitionTime': '2026-07-20T11:09:53Z', 'severity': 'Info', 'status': 'True', 'type': 'WorkerWorkloadReady'}, {'lastTransitionTime': '2026-07-20T11:09:39Z', 'message': 'failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1"', 'reason': 'ScalingCRDNotFound', 'status': 'False', 'type': 'WorkloadsReady'}] [e2e-llm-inference-service] ---------------------------- Captured log teardown ----------------------------- [e2e-llm-inference-service] INFO e2e.llmisvc.namespace:namespace.py:97 Deleted test namespace e2e-test-llm-autoscaling-keda-lws-e541a132 [e2e-llm-inference-service] =============================== warnings summary =============================== [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_flow_control.py::test_flow_control_smoke[cluster_cpu-cluster_single_node-flow-control-utilization-detector] [e2e-llm-inference-service] /workspace/source/python/kserve/.venv/lib64/python3.11/site-packages/pytest_asyncio/plugin.py:761: DeprecationWarning: The event_loop fixture provided by pytest-asyncio has been redefined in [e2e-llm-inference-service] /workspace/source/test/e2e/conftest.py:43 [e2e-llm-inference-service] Replacing the event_loop fixture with a custom implementation is deprecated [e2e-llm-inference-service] and will lead to errors in the future. [e2e-llm-inference-service] If you want to request an asyncio event loop with a scope other than function [e2e-llm-inference-service] scope, use the "scope" argument to the asyncio mark when marking the tests. [e2e-llm-inference-service] If you want to return different types of event loops, use the event_loop_policy [e2e-llm-inference-service] fixture. [e2e-llm-inference-service] [e2e-llm-inference-service] warnings.warn( [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-custom-template-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_flow_control.py::test_flow_control_smoke[cluster_cpu-cluster_single_node-flow-control-utilization-detector] [e2e-llm-inference-service] llmisvc/test_flow_control.py:47: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_flow_control.py::test_flow_control_smoke[cluster_cpu-cluster_single_node-flow-control-concurrency-detector] [e2e-llm-inference-service] llmisvc/test_flow_control.py:47: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-pd-config-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-v06-nonzero-threshold-migration-workload-llmd-simulator-pd] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-precise-prefix-cache-inline-config-workload-llmd-simulator-kvcache] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator2] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf0] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m-with-lora-hf1] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-pvc] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py::test_llm_stop_feature[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service_stop.py:40: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_tls.py::test_llm_tls_resources[cluster_cpu-cluster_single_node-router-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_tls.py:92: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.llminferenceservice [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-scheduler-managed-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-custom-route-timeout-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-with-refs-pd-scheduler-managed-workload-pd-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-no-scheduler-workload-single-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_multi_node-router-managed-workload-simulated-dp-ep-cpu-model-fb-opt-125m] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-inline-config-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-model-qwen2.5-0.5b] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-configmap-ref-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py::test_llm_inference_service[cluster_cpu-cluster_single_node-router-managed-scheduler-with-replicas-workload-llmd-simulator] [e2e-llm-inference-service] llmisvc/test_llm_inference_service.py:243: PytestWarning: The test is marked with '@pytest.mark.asyncio' but it is not an async function. Please remove the asyncio mark. If the test is not marked explicitly, check for global marks applied via 'pytestmark'. [e2e-llm-inference-service] @pytest.mark.asyncio(loop_scope="session") [e2e-llm-inference-service] [e2e-llm-inference-service] -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html [e2e-llm-inference-service] ---------- generated xml file: /workspace/artifacts-dir/junit_e2e.xml ---------- [e2e-llm-inference-service] --------------------------------- JSON report ---------------------------------- [e2e-llm-inference-service] report saved to: /workspace/artifacts-dir/e2e_results.json [e2e-llm-inference-service] =========================== short test summary info ============================ [e2e-llm-inference-service] FAILED llmisvc/test_llm_auth.py::test_llm_auth_enabled_requires_token[cluster_cpu-cluster_single_node-auth-enabled-default] [e2e-llm-inference-service] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_hpa_deployment[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-hpa] [e2e-llm-inference-service] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_keda_deployment[cluster_cpu-cluster_single_node-router-managed-workload-llmd-simulator-no-replicas-prometheus-scrape-scaling-keda] [e2e-llm-inference-service] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_hpa_lws[cluster_cpu-cluster_multi_node-router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-hpa] [e2e-llm-inference-service] FAILED llmisvc/test_llm_autoscaling_wva.py::test_llm_autoscaling_keda_lws[cluster_cpu-cluster_multi_node-router-managed-workload-llmd-simulator-lws-prometheus-scrape-scaling-keda] [e2e-llm-inference-service] !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 5 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! [e2e-llm-inference-service] !!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 5 failures !!!!!!!!!!!!! [e2e-llm-inference-service] ====== 5 failed, 46 passed, 3 skipped, 28 warnings in 4490.62s (1:14:50) ======= [must-gather] [must-gather ] OUT 2026-07-20T11:25:49.342091696Z Using must-gather plug-in image: quay.io/modh/must-gather:rhoai-2.24 [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 18243c48-a001-4376-9d1d-4ce350941cb0 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.25" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-20T11:25:49.39484866Z namespace/openshift-must-gather-dwbhh created [must-gather] [must-gather ] OUT 2026-07-20T11:25:49.404242514Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-m9zfb created [must-gather] [must-gather ] OUT 2026-07-20T11:25:49.429969478Z pod for plug-in image quay.io/modh/must-gather:rhoai-2.24 created [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.240385825Z [disk usage checker] Started [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.243645613Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.516804863Z Error from server (NotFound): namespaces "redhat-ods-operator" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.694943559Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.694974977Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.694974977Z namespaces "redhat-ods-operator" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.698011412Z Error getting logs from redhat-ods-operator [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.867503821Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.867536781Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.867536781Z namespaces "redhat-ods-monitoring" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:56.870393530Z Error getting logs from redhat-ods-monitoring [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.046582136Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.046613061Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.046613061Z namespaces "redhat-ods-applications" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.049500880Z Error getting logs from redhat-ods-applications [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.218486747Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.218517261Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.218517261Z namespaces "rhods-notebooks" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.221155604Z Error getting logs from rhods-notebooks [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.390110804Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.390144650Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.390144650Z namespaces "rhoai-model-registries" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.392902803Z Error getting logs from rhoai-model-registries [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.569921836Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.569954593Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.569954593Z namespaces "istio-system" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.572972701Z Error getting logs from istio-system [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.739837451Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.739868460Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.739868460Z namespaces "knative-serving" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.742818517Z Error getting logs from knative-serving [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.909016183Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.909048677Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.909048677Z namespaces "redhat-ods-applications-auth-provider" not found [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:57.911865822Z Error getting logs from redhat-ods-applications-auth-provider [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.211121537Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.501940771Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.649609544Z error: the server doesn't have a resource type "auths" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.737425445Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.740277162Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.878683813Z error: the server doesn't have a resource type "monitorings" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.965803517Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:58.968470981Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.103678632Z error: the server doesn't have a resource type "featuretrackers" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.190986471Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.193617353Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.333397172Z error: the server doesn't have a resource type "codeflares" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.418118241Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.420686568Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.559022955Z error: the server doesn't have a resource type "dashboards" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.646776383Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.649457519Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.788977967Z error: the server doesn't have a resource type "datasciencepipelines" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.877963343Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:25:59.880860895Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.020243567Z error: the server doesn't have a resource type "feastoperators" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.106794181Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.109575977Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.248483731Z error: the server doesn't have a resource type "kserves" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.334946918Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.337663469Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.472721478Z error: the server doesn't have a resource type "kueues" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.561605753Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.564247715Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.700219313Z error: the server doesn't have a resource type "modelcontrollers" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.786750633Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.789730661Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:00.923947113Z error: the server doesn't have a resource type "modelmeshservings" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.006401719Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.009076604Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.147251574Z error: the server doesn't have a resource type "modelregistries" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.234965985Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.237498892Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.248292544Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.386766000Z error: the server doesn't have a resource type "rays" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.473947722Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.476766638Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.616497967Z error: the server doesn't have a resource type "trainingoperators" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.704064587Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.706985090Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.848769879Z error: the server doesn't have a resource type "trustyais" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.938460209Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:01.941308682Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.087746726Z error: the server doesn't have a resource type "workbenches" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.175625582Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.178428072Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.316790988Z error: the server doesn't have a resource type "hardwareprofiles" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.406446891Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.409183343Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.552640914Z error: the server doesn't have a resource type "llamastackoperators" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.648300122Z error: arguments in resource/name form must have a single resource and name [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:02.651114597Z Error collecting info from [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:03.405608163Z error: the server doesn't have a resource type "predictors" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:03.546943101Z error: the server doesn't have a resource type "localmodelnodegroups" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:03.683828878Z error: the server doesn't have a resource type "smcp" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:03.821545071Z error: the server doesn't have a resource type "smm" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:03.958973078Z error: the server doesn't have a resource type "smmr" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:04.621384171Z error: the server doesn't have a resource type "knativeservings" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:04.755919974Z error: the server doesn't have a resource type "configurations" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:04.897416846Z error: the server doesn't have a resource type "routes" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:05.037598880Z error: the server doesn't have a resource type "services" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:05.171718117Z error: the server doesn't have a resource type "revisions" [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:05.721067490Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:05.860994580Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:05.955029332Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.028582192Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.198341832Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.254095412Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.339796009Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.442480387Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.524028580Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.692059181Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.836826986Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:06.925999170Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.001939572Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.170973686Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.306800047Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.389452694Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.462150731Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.640581386Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.794750589Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.899734180Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:07.990675291Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.160604913Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.308912940Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.395358213Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.469161025Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.641334035Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.781594666Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.867062279Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:08.941791090Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.113426415Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.250337148Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.332721689Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.404634895Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.577321975Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.724890339Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.811669742Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:09.884136255Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.053872975Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.191290947Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.275574468Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.356164989Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.523535494Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.664325345Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.750358032Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.821999985Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:10.992819451Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.127196814Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.211545649Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.261672873Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.291813820Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.465167667Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.605925358Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.689842254Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.762858676Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:11.933481368Z Gathering data for ns/kserve-ci-e2e-test... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:12.070473427Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:12.153998977Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:12.223168773Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:12.392051114Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:12.763433153Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:12.922475445Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:13.033346680Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:13.203265620Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:13.597578840Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:13.757689111Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:13.872589623Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:14.043364414Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:14.574414531Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:14.776121486Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:14.900668892Z Wrote inspect data to must-gather. [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:16.266372330Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 [must-gather] [must-gather-sjl6j] POD 2026-07-20T11:26:18.815867102Z Caches written to disk [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:19.824712608Z waiting for gather to complete [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:19.835213573Z downloading gather output [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.148421909Z receiving incremental file list [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.162706388Z ./ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.16277798Z aggregated-discovery-api.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.163098846Z aggregated-discovery-apis.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.164127896Z event-filter.html [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.164537394Z timestamp [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.164671317Z version [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169545692Z cluster-scoped-resources/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169559782Z cluster-scoped-resources/datasciencecluster.opendatahub.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169565813Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169575183Z cluster-scoped-resources/datasciencecluster.opendatahub.io/datascienceclusters/test-dsc.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169660734Z cluster-scoped-resources/dscinitialization.opendatahub.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169675555Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169699105Z cluster-scoped-resources/dscinitialization.opendatahub.io/dscinitializations/test-dsci.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169770026Z namespaces/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169781127Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169801897Z namespaces/kserve-ci-e2e-test/kserve-ci-e2e-test.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169872648Z namespaces/kserve-ci-e2e-test/apps.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169915739Z namespaces/kserve-ci-e2e-test/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.16995834Z namespaces/kserve-ci-e2e-test/apps/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.169999821Z namespaces/kserve-ci-e2e-test/apps/daemonsets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170084783Z namespaces/kserve-ci-e2e-test/apps/deployments.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170162264Z namespaces/kserve-ci-e2e-test/apps/replicasets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170265896Z namespaces/kserve-ci-e2e-test/apps/statefulsets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170313247Z namespaces/kserve-ci-e2e-test/autoscaling/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170357978Z namespaces/kserve-ci-e2e-test/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170415749Z namespaces/kserve-ci-e2e-test/batch/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.17045771Z namespaces/kserve-ci-e2e-test/batch/cronjobs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170538872Z namespaces/kserve-ci-e2e-test/batch/jobs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170585932Z namespaces/kserve-ci-e2e-test/build.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170626513Z namespaces/kserve-ci-e2e-test/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170707695Z namespaces/kserve-ci-e2e-test/build.openshift.io/builds.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170754096Z namespaces/kserve-ci-e2e-test/core/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170795766Z namespaces/kserve-ci-e2e-test/core/configmaps.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.170919599Z namespaces/kserve-ci-e2e-test/core/endpoints.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.17100243Z namespaces/kserve-ci-e2e-test/core/events.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171077532Z namespaces/kserve-ci-e2e-test/core/persistentvolumeclaims.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171161624Z namespaces/kserve-ci-e2e-test/core/pods.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171260915Z namespaces/kserve-ci-e2e-test/core/replicationcontrollers.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171381408Z namespaces/kserve-ci-e2e-test/core/secrets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171564901Z namespaces/kserve-ci-e2e-test/core/services.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171615542Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171658863Z namespaces/kserve-ci-e2e-test/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171706164Z namespaces/kserve-ci-e2e-test/image.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171749025Z namespaces/kserve-ci-e2e-test/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171797446Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171838717Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171918008Z namespaces/kserve-ci-e2e-test/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.171965119Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.17200559Z namespaces/kserve-ci-e2e-test/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172050261Z namespaces/kserve-ci-e2e-test/networking.k8s.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172093872Z namespaces/kserve-ci-e2e-test/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172138063Z namespaces/kserve-ci-e2e-test/policy/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172178293Z namespaces/kserve-ci-e2e-test/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172236835Z namespaces/kserve-ci-e2e-test/route.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172290676Z namespaces/kserve-ci-e2e-test/route.openshift.io/routes.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172334876Z namespaces/kuadrant-system/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172377967Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172430798Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172470589Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.17251771Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172558141Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172637673Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.172986899Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173328116Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173361767Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173409628Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173456298Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173496469Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173577241Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173625072Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173653982Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173743574Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173790385Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173829696Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.173971639Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.174100731Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.174660352Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.174736213Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175119821Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175256184Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175474928Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.17561028Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175647851Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175772644Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175801374Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175855915Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175885076Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.175975607Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176004558Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176056129Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.17609294Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176143631Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176211752Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176223923Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176259613Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino-7c77957575-mjgp9.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176334285Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176342095Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176346635Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.176381146Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181517506Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181586157Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181614398Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181669099Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/authorino-operator-6d85f6564-kxw42.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181767861Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181777981Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181783521Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181791661Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.181986965Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182054346Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182080937Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182122928Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/dns-operator-controller-manager-65b49595d7-ld9rx.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.18224594Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.18226313Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182271161Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182283711Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182397553Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182462264Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182513425Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182525295Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin-85fd4b8b9d-7fxgf.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182618977Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182628248Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182634938Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182655718Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.18273624Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182802151Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182830172Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182875012Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182990165Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.182998815Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.183005095Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.183019635Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.189775237Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.189847329Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.189870099Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.18992626Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador-limitador-69574b596d-q97v8.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190006232Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190015152Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190020302Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190039722Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190138244Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190223396Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190243736Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190302807Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/limitador-operator-controller-manager-6f9f468797-rvhvm.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190398469Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190407109Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190411049Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.19043488Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190579093Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190643954Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190680115Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190707725Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190763836Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190806457Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190858058Z namespaces/openshift-ingress/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.190904459Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.19095679Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191003641Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191050662Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191083213Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191185495Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.19148357Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191687914Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191722225Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191777476Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191844168Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191866668Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.19195991Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.191998731Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.192044332Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.192124923Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.192183124Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.192269766Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.192698894Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.192798046Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.193183224Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.193263975Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.19350235Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.193635492Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.193831406Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.193918688Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.193955799Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.19403598Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194072901Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194107842Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194161803Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194266805Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194316076Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194352766Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194419938Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194456219Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194501119Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194508549Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.19454901Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/istiod-openshift-gateway-94bb8fbfd-zltzb.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194657143Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194668183Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194673313Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.194680773Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204337241Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204405863Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204438343Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204484114Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204599006Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204610707Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204615617Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.204623307Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.20530906Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205407032Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205453473Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router-default-5f6bfd579b-gk6x5.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205552365Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205559525Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205563625Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205588236Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/current.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205707338Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/previous.insecure.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205774629Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/previous.log [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.20581169Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205844111Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205893472Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.205936123Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.214211534Z [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.214226295Z sent 2,836 bytes received 684,285 bytes 458,080.67 bytes/sec [must-gather] [must-gather-sjl6j] OUT 2026-07-20T11:26:20.214237815Z total size is 8,713,119 speedup is 12.68 [must-gather] [must-gather ] OUT 2026-07-20T11:26:20.266336442Z namespace/openshift-must-gather-dwbhh deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 18243c48-a001-4376-9d1d-4ce350941cb0 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.25" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-20T11:26:20.386734484Z Using must-gather plug-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ebc3605e5473a4c43fbc08ec8b84ef841febee8485b7f07cbcf2485e97a25ec6 [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 18243c48-a001-4376-9d1d-4ce350941cb0 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.25" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [must-gather] [must-gather ] OUT 2026-07-20T11:26:20.407763394Z namespace/openshift-must-gather-4rtcj created [must-gather] [must-gather ] OUT 2026-07-20T11:26:20.414888423Z clusterrolebinding.rbac.authorization.k8s.io/must-gather-nfx2d created [must-gather] [must-gather ] OUT 2026-07-20T11:26:20.433330524Z pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ebc3605e5473a4c43fbc08ec8b84ef841febee8485b7f07cbcf2485e97a25ec6 created [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:22.047648298Z [disk usage checker] Started [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:22.051248941Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:22.262889957Z Gathering data for ns/openshift-cluster-version... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:22.421293261Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:22.526928169Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:22.624098913Z Gathering data for ns/default... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:22.975770049Z Gathering data for ns/openshift... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:23.428754814Z Gathering data for ns/kube-system... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:24.210614114Z Gathering data for ns/openshift-etcd... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:24.558561919Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:24.558593660Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:24.558593660Z namespaces "assisted-installer" not found [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.047894401Z Waiting on subprocesses to finish execution. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.056796829Z INFO: Collecting host service logs for crio [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.057125927Z INFO: Collecting host service logs for kubelet [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.057438256Z INFO: Collecting host service logs for rpm-ostreed [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.058058046Z INFO: Collecting host service logs for ostree-finalize-staged [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.058413333Z INFO: Collecting host service logs for machine-config-daemon-firstboot [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.058726607Z INFO: Gathering HAProxy config files [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.058753991Z INFO: Collecting host service logs for machine-config-daemon-host [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.059100872Z INFO: Collecting host service logs for NetworkManager [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.059565059Z INFO: Collecting host service logs for openvswitch [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.059895973Z INFO: Collecting host service logs for ovs-configuration [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.060216756Z INFO: Collecting host service logs for ovsdb-server [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.060568235Z INFO: Collecting host service logs for ovs-vswitchd [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.060926530Z INFO: Waiting for worker host service log collection to complete ... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.061341171Z INFO: Gathering machine config daemon's old logs from all nodes [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.062739035Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.075401639Z Executing Istio gather script [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.077400090Z INFO: Gathering on-disk MachineConfig from degraded nodes [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.118255240Z INFO: Waiting for node performance related collection to complete ... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.727799324Z INFO: "sriov-network-operator" not detected. Skipping. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.768929389Z No resources found in openshift-etcd namespace. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:25.896209245Z INFO: Found 1 replicas - prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.048446120Z INFO: namespace openshift-frr-k8s not detected. Skipping. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.115413859Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.127786785Z INFO: Waiting for on-disk MachineConfig collection to complete ... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.127817266Z INFO: on-disk MachineConfig config collection complete. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.212161145Z INFO: "metallb-operator" not detected. Skipping. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.508948480Z No resources found [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.566701605Z INFO: Getting alertmanagers from prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.640687857Z INFO: Waiting for HAProxy config collection to complete ... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.699482278Z INFO: Worker host service log collection to complete. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.756984214Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-rm45p [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.757046733Z insights-runtime-extractor-fqs2h [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.757071788Z insights-runtime-extractor-h8tvt [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.757092902Z insights-runtime-extractor-j7t54 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.894373123Z error: the server doesn't have a resource type "performanceprofile" [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.922399801Z ERROR: No running kube-apiserver pods found [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:26.978236241Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.194935856Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.250369167Z tar: Removing leading `/' from member names [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.263080622Z INFO: HAProxy config collection complete. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.514734520Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.514734520Z See 'oc rsync -h' for help and examples [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.565375906Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.733418945Z INFO: 'previous-logs' folder not found on ip-10-0-133-160.ec2.internal, skipping... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.765783870Z No resources found [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:27.851429524Z error: the server doesn't have a resource type "multi-networkpolicy" [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.037738567Z Gathering data for ns/kuadrant-system... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.146514561Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.361677569Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.368902452Z INFO: Getting rules from prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.477399693Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.522586610Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.595674637Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istio-reader-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.619428301Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.871216208Z error: the server doesn't have a resource type "machineconfigs" [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.977032541Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:28.978846048Z INFO: 'previous-logs' folder not found on ip-10-0-137-140.ec2.internal, skipping... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.080185401Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.205024402Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.214172718Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istiod-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.245823213Z error: the server doesn't have a resource type "machineconfigpools" [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.276420790Z INFO: Getting status/config from prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.544673979Z W0720 11:26:29.544622 1146 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.552083682Z INFO: 'previous-logs' folder not found on ip-10-0-138-196.ec2.internal, skipping... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.552120861Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.552154282Z INFO: Machine Config Daemon termination log collection complete. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.588492256Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.615428782Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.633920599Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.818239900Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.821631774Z Inspecting resource clusterrole.rbac.authorization.k8s.io/istiod-gateway-controller-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.841867405Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:29.951215889Z Gathering data for ns/openshift-monitoring... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.103626586Z error: the server doesn't have a resource type "kubeletconfigs" [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.118186804Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.135219637Z INFO: Getting status/flags from prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.232865226Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.244902249Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istio-reader-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.475085566Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.526913972Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.585333109Z INFO: INTERCONNECT MODE [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.585373984Z INFO: Gathering ovn-kubernetes DBs [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.620166299Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.623733863Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istiod-clusterrole-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.821249371Z INFO: Getting targets?state=active from prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.837329399Z INFO: Gathering OVN_Northbound from ovnkube-node-8h6hb... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.837672666Z INFO: Gathering OVN_Northbound from ovnkube-node-h4csm... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.837950460Z INFO: Gathering OVN_Northbound from ovnkube-node-kg8lz... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.838252223Z INFO: Gathering OVN_Southbound from ovnkube-node-8h6hb... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.838714572Z INFO: Gathering OVN_Southbound from ovnkube-node-h4csm... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.839080618Z INFO: Gathering OVN_Southbound from ovnkube-node-kg8lz... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:30.958492776Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.064840220Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.081800019Z Inspecting resource clusterrolebinding.rbac.authorization.k8s.io/istiod-gateway-controller-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.162073054Z tar: Removing leading `/' from member names [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.181214549Z tar: Removing leading `/' from member names [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.192429553Z tar: Removing leading `/' from member names [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.245012330Z INFO: Getting status/tsdb from prometheus-k8s-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.314275946Z tar: Removing leading `/' from member names [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.354734899Z tar: Removing leading `/' from member names [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.365440534Z tar: Removing leading `/' from member names [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.506127516Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.521915024Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.853374723Z INFO: Getting status from alertmanager-main-0 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:31.903346164Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.118856521Z INFO: Waiting for network log collection to complete ... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.118891108Z INFO: Waiting for ovnk database copies to complete ... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.119404946Z INFO: Copying ovnk databases complete. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.121590960Z 7.3M must-gather/network_logs/ovnk_database_store [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.124203784Z ovnk_database_store/ [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.124203784Z ovnk_database_store/ovnkube-node-kg8lz_nbdb [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.141648905Z ovnk_database_store/ovnkube-node-h4csm_nbdb [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.157601369Z ovnk_database_store/ovnkube-node-h4csm_sbdb [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.214481270Z ovnk_database_store/ovnkube-node-8h6hb_nbdb [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.222129472Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.230112654Z ovnk_database_store/ovnkube-node-kg8lz_sbdb [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.235134948Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.274677829Z ovnk_database_store/ovnkube-node-8h6hb_sbdb [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.276411109Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.320859944Z INFO: Network log collection complete. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.609309252Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.658551664Z Gathering data for ns/openshift-network-console... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:32.958794793Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:33.188447340Z Gathering data for ns/openshift-console-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:33.280939933Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:33.283807820Z INFO: Fallback to identify the container image from release info [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:33.643458691Z Gathering data for ns/openshift-console... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:33.710948485Z Inspecting resource crd/authorizationpolicies.security.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:33.928319046Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:33.934634280Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.086762792Z daemonset.apps/perf-node-gather-daemonset created [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.185716943Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.189405316Z Inspecting resource crd/destinationrules.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.273862384Z Waiting for performance profile collector pods to become ready: 1 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.307109775Z Gathering data for ns/openshift-cluster-storage-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.465803635Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.722902923Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.725923990Z Inspecting resource crd/envoyfilters.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.942795410Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:34.950627250Z Gathering data for ns/openshift-dns-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.183809219Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.186906160Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.409323337Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.451193724Z Gathering data for ns/openshift-dns... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.454403886Z Waiting for performance profile collector pods to become ready: 2 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.627677381Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.630077198Z Inspecting resource crd/gateways.gateway.networking.k8s.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:35.836066679Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.065666620Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.068807000Z Inspecting resource crd/gateways.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.157247533Z Gathering data for ns/openshift-image-registry... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.270729545Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.502499119Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.505090155Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.604193869Z Waiting for performance profile collector pods to become ready: 3 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.735874903Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.771242067Z Gathering data for ns/openshift-ingress-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.968487264Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:36.970943947Z Inspecting resource crd/httproutes.gateway.networking.k8s.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.190144455Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.225412398Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.229399644Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.468486237Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.472712293Z Inspecting resource crd/inferencepools.inference.networking.k8s.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.672416863Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.771008898Z Waiting for performance profile collector pods to become ready: 4 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.894371418Z Gathering data for ns/openshift-ingress-canary... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.925766500Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:37.928379279Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.163673000Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.424657977Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.428095634Z Inspecting resource crd/peerauthentications.security.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.452345489Z Gathering data for ns/openshift-insights... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.601688435Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.832333192Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.834989786Z Inspecting resource crd/proxyconfigs.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:38.921810342Z Waiting for performance profile collector pods to become ready: 5 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:39.028909549Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:39.263956220Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:39.266706534Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:39.434801776Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:39.630180324Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:39.632476958Z Inspecting resource crd/requestauthentications.security.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:39.809184230Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.025186810Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.028275197Z Inspecting resource crd/serviceentries.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.067786827Z Waiting for performance profile collector pods to become ready: 6 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.222165255Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.427911656Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.429942371Z Inspecting resource crd/sidecars.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.621403032Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.832281665Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:40.834713932Z Inspecting resource crd/telemetries.telemetry.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.025804973Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.215677559Z Waiting for performance profile collector pods to become ready: 7 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.250235836Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.253177543Z Inspecting resource crd/virtualservices.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.474198789Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.684898862Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.688142351Z Inspecting resource crd/wasmplugins.extensions.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.728368861Z Gathering data for ns/openshift-lws-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:41.882123929Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.108370758Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.111761023Z Inspecting resource crd/workloadentries.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.235788957Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.327517877Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.365340661Z Daemonset perf-node-gather-daemonset ready 3 out of 3 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.391660291Z Gathering data for ns/kserve... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.557264337Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.560184175Z Inspecting resource crd/workloadgroups.networking.istio.io [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.682303118Z Collecting performance related data for node ip-10-0-138-196.ec2.internal [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.691870667Z Collecting performance related data for node ip-10-0-137-140.ec2.internal [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.705922710Z Collecting performance related data for node ip-10-0-133-160.ec2.internal [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:42.820206985Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:43.116006753Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:43.273312697Z Inspecting resource mutatingwebhookconfiguration.admissionregistration.k8s.io/istio-sidecar-injector-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:43.534477909Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:43.979013591Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:44.211306259Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:44.385479213Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:44.557116587Z Inspecting resource validatingwebhookconfiguration.admissionregistration.k8s.io/istio-validator-openshift-gateway-openshift-ingress [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:44.797174945Z Gathering data for ns/openshift-ingress... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:45.206227165Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:45.210545824Z Gathering data for ns/openshift-config... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:45.477556461Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:45.606006380Z Gathering data for ns/openshift-config-managed... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:45.648865177Z Wrote inspect data to /must-gather/istio. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:46.232156790Z Gathering data for ns/openshift-kube-apiserver-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:46.249114479Z Done executing Istio gather script [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:46.570258196Z Gathering data for ns/openshift-kube-apiserver... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:46.875636451Z Gathering data for ns/openshift-kube-controller-manager... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:47.244362538Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:47.259640477Z Gathering data for ns/openshift-kube-controller-manager-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:47.625277863Z Gathering data for ns/openshift-kube-scheduler... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:47.964434070Z Gathering data for ns/openshift-kube-scheduler-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:48.361505890Z Gathering data for ns/openshift-kube-storage-version-migrator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:48.758662233Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:49.138012579Z Gathering data for ns/openshift-user-workload-monitoring... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:49.714943215Z Gathering data for ns/openshift-multus... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:51.278062582Z Gathering data for ns/openshift-ovn-kubernetes... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:51.787962893Z Collecting kubelet logs for node ip-10-0-138-196.ec2.internal [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:51.789967501Z Collecting kubelet logs for node ip-10-0-137-140.ec2.internal [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:51.792412810Z Collecting kubelet logs for node ip-10-0-133-160.ec2.internal [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:52.232320629Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4rtcj namespace [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:52.237198702Z INFO: Node performance data collection complete. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:52.250113287Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:52.783962519Z Gathering data for ns/openshift-host-network... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:53.125867804Z Gathering data for ns/openshift-network-diagnostics... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:53.777099925Z Gathering data for ns/openshift-network-node-identity... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:54.259866366Z Gathering data for ns/openshift-network-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:54.666421945Z Gathering data for ns/openshift-cloud-network-config-controller... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:54.966880381Z Gathering data for ns/openshift-cluster-node-tuning-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:55.492800639Z Gathering data for ns/openshift-apiserver-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:55.808539731Z Gathering data for ns/openshift-apiserver... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:56.263194080Z Gathering data for ns/openshift-controller-manager-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:56.564269090Z Gathering data for ns/openshift-controller-manager... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:56.874915315Z Gathering data for ns/openshift-cluster-samples-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:57.255837716Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:57.574327190Z Gathering data for ns/openshift-operator-lifecycle-manager... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:57.895880070Z Gathering data for ns/openshift-service-ca-operator... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:58.259880613Z Gathering data for ns/openshift-service-ca... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:58.684092402Z Gathering data for ns/openshift-cluster-csi-drivers... [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:59.704521576Z Wrote inspect data to must-gather. [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:59.704615895Z error: inspection completed with the errors occurred while gathering data: [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:59.704615895Z skipping gathering secrets/support due to error: secrets "support" not found [must-gather] [must-gather-xsljk] POD 2026-07-20T11:26:59.843997222Z error: the server doesn't have a resource type "clusters" [must-gather] [must-gather-xsljk] POD 2026-07-20T11:27:00.015549945Z Caches written to disk [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.023846674Z waiting for gather to complete [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.026762071Z downloading gather output [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.310400343Z receiving incremental file list [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.331875698Z ./ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.33197026Z aggregated-discovery-api.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.332280976Z aggregated-discovery-apis.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.333550361Z event-filter.html [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.336724564Z timestamp [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.336799566Z version [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345298394Z cluster-scoped-resources/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345322114Z cluster-scoped-resources/admissionregistration.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345332154Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345361415Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/default-network-annotation.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345509498Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/servicecidrs.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.34561809Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicies/user-defined-networks-namespace-label.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345698302Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345733142Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/default-network-annotation-binding.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345835935Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/servicecidrs-binding.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.345921806Z cluster-scoped-resources/admissionregistration.k8s.io/validatingadmissionpolicybindings/user-defined-networks-namespace-label-binding.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.346028408Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.346078459Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/multus.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.346226132Z cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/network-node-identity.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.346295953Z cluster-scoped-resources/apiextensions.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.346787723Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.346829564Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/accounts.nim.opendatahub.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.347026188Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.347438826Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/adminpolicybasedexternalroutes.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.34764961Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertingrules.monitoring.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.347800133Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagerconfigs.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.351533147Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertmanagers.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.353050207Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/alertrelabelconfigs.monitoring.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.353254971Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeyapprovals.devportal.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.353367244Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeyrequests.devportal.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.353483316Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apikeys.devportal.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.353616318Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiproducts.devportal.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.354102218Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apirequestcounts.apiserver.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.354294672Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/apiservers.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.354466915Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/appliedmanifestworks.work.open-cluster-management.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.354583918Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authconfigs.authorino.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.355385664Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authentications.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.355614388Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorinos.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.3557409Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorizationpolicies.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.355906194Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authpolicies.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.357152708Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/baselineadminnetworkpolicies.policy.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.357400433Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/builds.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.357562887Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/bundles.trust.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.35772702Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/catalogsources.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.35873173Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificaterequests.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.358897733Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certificates.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.359133247Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/certmanagers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.359483714Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/challenges.acme.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.360221809Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudcredentials.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.360342822Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.360463114Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/cloudprivateipconfigs.cloud.network.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.360569356Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterclaims.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.360669168Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercloudeventsources.eventing.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.36077578Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustercsidrivers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.360933763Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterimagepolicies.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.361114177Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterissuers.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.361920563Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteroperators.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.362060916Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterresourcequotas.quota.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.362193408Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterserviceversions.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.364225658Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterstoragecontainers.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.364405702Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clustertriggerauthentications.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.364555785Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusteruserdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.36480656Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/clusterversions.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.365086605Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.365543654Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.365651936Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/configs.samples.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.365774309Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleclidownloads.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.365886771Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleexternalloglinks.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.365996883Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolelinks.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.366107756Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolenotifications.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.366246428Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleplugins.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.366426162Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolequickstarts.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.366563845Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.366668547Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoles.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.366935802Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consolesamples.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.367086925Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/consoleyamlsamples.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.367238968Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/controllerconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.367810939Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/credentialsrequests.cloudcredential.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.367926461Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/csisnapshotcontrollers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.368048634Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/datascienceclusters.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.368267558Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/destinationrules.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.369009873Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.369127345Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnses.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.369331629Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnshealthcheckprobes.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.369441782Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnspolicies.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.369611295Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.ingress.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.369723757Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dnsrecords.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.36987824Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/dscinitializations.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.370010313Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressfirewalls.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.370127215Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressips.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.370265028Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressqoses.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.37040223Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressrouters.network.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.370528843Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/egressservices.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.370666946Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/envoyfilters.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.370799448Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/etcds.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.370938631Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/featuregates.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.371078784Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.371238517Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.371609794Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.371785618Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/grpcroutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.373478442Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/helmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.373604234Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httproutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.374455671Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httpscaledobjects.http.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.374590693Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentpolicies.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.374713086Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagecontentsourcepolicies.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.374820038Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagedigestmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.37494092Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepolicies.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.375144084Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagepruners.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.375452381Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/images.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.375580163Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/imagetagmirrorsets.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.375694025Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencegraphs.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.375850138Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencemodelrewrites.llm-d.ai.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.375958911Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceobjectives.llm-d.ai.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.376081763Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.376254236Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.376384799Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.380298776Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/infrastructures.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.380740315Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresscontrollers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.381359847Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ingresses.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.381545001Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/insightsoperators.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.381711984Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/installplans.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.381851647Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/interceptorroutes.http.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.38199085Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ipamclaims.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.382099792Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ippools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.382224874Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/issuers.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.383068411Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/istiocsrs.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.383414398Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kedacontrollers.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.387903127Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kuadrants.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.388031309Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeapiservers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.388191893Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubecontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.388330795Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubeschedulers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.388457178Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/kubestorageversionmigrators.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.38856731Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersetoperators.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.388693583Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/leaderworkersets.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.391345465Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/limitadors.limitador.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.396891515Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceserviceconfigs.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.401633728Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/llminferenceservices.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.406159138Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfignodes.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.406344892Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineconfigurations.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.406646408Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosbuilds.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.406840351Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/machineosconfigs.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.406983954Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/network-attachment-definitions.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.407093896Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.407310081Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/networks.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.407574466Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodes.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.407689838Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/nodeslicepools.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.407797561Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oauths.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.407991394Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/oidcpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.408123077Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/olmconfigs.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.40828177Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftapiservers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.408404533Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/openshiftcontrollermanagers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.408513205Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorconditions.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.408664998Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorgroups.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.40878838Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorhubs.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.408911363Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operatorpkis.network.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.409020635Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/operators.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.409386682Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/orders.acme.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.409410842Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/overlappingrangeipreservations.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.409451693Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/peerauthentications.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.409599476Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/pinnedimagesets.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.409705918Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/planpolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.409836911Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podmonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.410129156Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/podnetworkconnectivitychecks.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.41030486Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/probes.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.410577615Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/profiles.tuned.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.410710848Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projecthelmchartrepositories.helm.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.41082562Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/projects.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.410929312Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheuses.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.413121356Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/prometheusrules.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.413287289Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxies.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.413400081Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxyconfigs.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.413514903Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rangeallocations.security.internal.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.413621516Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.413773299Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.413892901Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/requestauthentications.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.414049894Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/rolebindingrestrictions.authorization.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.414198107Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledjobs.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.415980892Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/scaledobjects.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.416134465Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/schedulers.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.416298658Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/securitycontextconstraints.security.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.416453682Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicecas.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.416568534Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serviceentries.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.416757518Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servicemonitors.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.417006403Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/servingruntimes.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.417548443Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/sidecars.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.417826569Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storages.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.417943411Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storagestates.migration.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.418049553Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/storageversionmigrations.migration.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.418185046Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/subscriptions.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.418850759Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetries.telemetry.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.419035783Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetrypolicies.extensions.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.419153375Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/thanosrulers.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.420586703Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tlspolicies.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.420729486Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tokenratelimitpolicies.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.420881569Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trainedmodels.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.420986701Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/triggerauthentications.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.421192415Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/trustmanagers.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.421481871Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/tuneds.tuned.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.421670715Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/userdefinednetworks.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.421835558Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/virtualservices.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.422264497Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumepopulators.populator.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.422377039Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotclasses.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.422515602Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshotcontents.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.422689645Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/volumesnapshots.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.422844008Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/wasmplugins.extensions.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.422973451Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadentries.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.423109664Z cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadgroups.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.423297047Z cluster-scoped-resources/apiregistration.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.423551212Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.423605163Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1..yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.423701485Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.acme.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.423781627Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.423874668Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiextensions.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.42396358Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apiserver.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424121633Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424259376Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.apps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424343118Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authentication.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.42443644Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424526492Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.authorization.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424619683Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.autoscaling.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424705235Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.batch.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424794027Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.build.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.424888539Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.42497487Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.certificates.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425067712Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloud.network.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425153784Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.cloudcredential.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425310437Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.config.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425386678Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.console.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425486051Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.coordination.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425579312Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.datasciencecluster.opendatahub.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425668034Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.discovery.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425755856Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.dscinitialization.opendatahub.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425851948Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.events.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.425936219Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.flowcontrol.apiserver.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426026931Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426138243Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.image.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426232365Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.imageregistry.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426320017Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.inference.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426428999Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.ingress.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426507851Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426610753Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.k8s.ovn.org.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426702495Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426786586Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.leaderworkerset.x-k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.426875308Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.machineconfiguration.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.42696624Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427054071Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.monitoring.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427143683Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.network.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427256125Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427344687Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427432989Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.nim.opendatahub.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427517711Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.node.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427608002Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.oauth.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427698634Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427789006Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427874658Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.packages.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.427971589Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.policy.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428089272Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.project.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428180464Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.quota.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428299666Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.rbac.authorization.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428386068Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.resource.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.42847982Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.route.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428578902Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.samples.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428662653Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.scheduling.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428757145Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.internal.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428844347Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.428939949Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.security.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429037151Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.snapshot.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429127272Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429236755Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.telemetry.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429324156Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.template.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429418258Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.tuned.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.42951269Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.user.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429600672Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1.work.open-cluster-management.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429803936Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.cluster.open-cluster-management.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429898548Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.controlplane.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.429983869Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.devportal.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430078931Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.eventing.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430182583Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430314426Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.extensions.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430422628Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.http.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43051012Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.k8s.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430599812Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430695903Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430781725Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.limitador.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430869947Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.migration.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.430964639Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43105139Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operator.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431144012Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431277425Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.policy.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431385917Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431478009Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.telemetry.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431564871Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.trust.cert-manager.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431660322Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha1.whereabouts.cni.cncf.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431749134Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431836176Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.llm-d.ai.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.431933158Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43202246Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha2.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432116832Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1alpha3.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432232204Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.admissionregistration.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432325076Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.external.metrics.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432437398Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43253109Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.helm.openshift.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432629072Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.http.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432715934Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432811065Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.metrics.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.432902417Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.monitoring.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433000449Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433086791Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.operator.authorino.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433187703Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.populator.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433298575Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433384317Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.serving.kserve.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433475149Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta1.storage.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43357345Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v1beta3.authorino.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433660252Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.autoscaling.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433754594Z cluster-scoped-resources/apiregistration.k8s.io/apiservices/v2.operators.coreos.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433804595Z cluster-scoped-resources/certificates.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433821325Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433880017Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-4dtf5.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.433982168Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-rjvrv.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43407199Z cluster-scoped-resources/certificates.k8s.io/certificatesigningrequests/system:openshift:openshift-monitoring-xblvt.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.434264784Z cluster-scoped-resources/config.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.434316195Z cluster-scoped-resources/config.openshift.io/apiservers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.434478498Z cluster-scoped-resources/config.openshift.io/authentications.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.434602861Z cluster-scoped-resources/config.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.434685242Z cluster-scoped-resources/config.openshift.io/clusterimagepolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.434767744Z cluster-scoped-resources/config.openshift.io/clusteroperators.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43508184Z cluster-scoped-resources/config.openshift.io/clusterversions.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435205983Z cluster-scoped-resources/config.openshift.io/consoles.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435296694Z cluster-scoped-resources/config.openshift.io/dnses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435391567Z cluster-scoped-resources/config.openshift.io/featuregates.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435506579Z cluster-scoped-resources/config.openshift.io/imagecontentpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43558747Z cluster-scoped-resources/config.openshift.io/imagedigestmirrorsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435693822Z cluster-scoped-resources/config.openshift.io/imagepolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435774964Z cluster-scoped-resources/config.openshift.io/images.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435862636Z cluster-scoped-resources/config.openshift.io/imagetagmirrorsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.435946698Z cluster-scoped-resources/config.openshift.io/infrastructures.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436039919Z cluster-scoped-resources/config.openshift.io/ingresses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436133741Z cluster-scoped-resources/config.openshift.io/networks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436249244Z cluster-scoped-resources/config.openshift.io/nodes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436334155Z cluster-scoped-resources/config.openshift.io/oauths.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436419427Z cluster-scoped-resources/config.openshift.io/operatorhubs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436503989Z cluster-scoped-resources/config.openshift.io/projects.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43658888Z cluster-scoped-resources/config.openshift.io/proxies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436679002Z cluster-scoped-resources/config.openshift.io/schedulers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436800704Z cluster-scoped-resources/config.openshift.io/clusteroperators/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436848135Z cluster-scoped-resources/config.openshift.io/clusteroperators/console.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.436948767Z cluster-scoped-resources/config.openshift.io/clusteroperators/csi-snapshot-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437035619Z cluster-scoped-resources/config.openshift.io/clusteroperators/dns.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437124901Z cluster-scoped-resources/config.openshift.io/clusteroperators/image-registry.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437238133Z cluster-scoped-resources/config.openshift.io/clusteroperators/ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437328125Z cluster-scoped-resources/config.openshift.io/clusteroperators/insights.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437424667Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43757648Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437659021Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-scheduler.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437754463Z cluster-scoped-resources/config.openshift.io/clusteroperators/kube-storage-version-migrator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437843695Z cluster-scoped-resources/config.openshift.io/clusteroperators/monitoring.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.437943497Z cluster-scoped-resources/config.openshift.io/clusteroperators/network.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43807266Z cluster-scoped-resources/config.openshift.io/clusteroperators/node-tuning.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438180831Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438281584Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438368095Z cluster-scoped-resources/config.openshift.io/clusteroperators/openshift-samples.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438471287Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-catalog.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438555739Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager-packageserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438645151Z cluster-scoped-resources/config.openshift.io/clusteroperators/operator-lifecycle-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438742413Z cluster-scoped-resources/config.openshift.io/clusteroperators/service-ca.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438829184Z cluster-scoped-resources/config.openshift.io/clusteroperators/storage.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438905776Z cluster-scoped-resources/config.openshift.io/clusterversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.438953857Z cluster-scoped-resources/config.openshift.io/clusterversions/version.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439031278Z cluster-scoped-resources/config.openshift.io/consoles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439066819Z cluster-scoped-resources/config.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43913939Z cluster-scoped-resources/config.openshift.io/featuregates/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439178461Z cluster-scoped-resources/config.openshift.io/featuregates/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439276463Z cluster-scoped-resources/config.openshift.io/infrastructures/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439324774Z cluster-scoped-resources/config.openshift.io/infrastructures/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439384315Z cluster-scoped-resources/config.openshift.io/oauths/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439422376Z cluster-scoped-resources/config.openshift.io/oauths/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439455217Z cluster-scoped-resources/config.openshift.io/proxies/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439509888Z cluster-scoped-resources/config.openshift.io/proxies/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439565919Z cluster-scoped-resources/console.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439585649Z cluster-scoped-resources/console.openshift.io/consoleplugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.43962017Z cluster-scoped-resources/console.openshift.io/consoleplugins/kuadrant-console-plugin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439712682Z cluster-scoped-resources/console.openshift.io/consoleplugins/monitoring-plugin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439788653Z cluster-scoped-resources/console.openshift.io/consoleplugins/networking-console-plugin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439850135Z cluster-scoped-resources/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439860025Z cluster-scoped-resources/core/nodes/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.439911906Z cluster-scoped-resources/core/nodes/ip-10-0-133-160.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440068259Z cluster-scoped-resources/core/nodes/ip-10-0-137-140.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440221292Z cluster-scoped-resources/core/nodes/ip-10-0-138-196.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440406916Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440414856Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440461137Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/catch-all.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440560819Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/exempt.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44064017Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/global-default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440724032Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440805063Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-scheduler.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.440891055Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/kube-system-service-accounts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441033888Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44112415Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver-sar.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441242852Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441326364Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-authentication-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441417406Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441508368Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441597789Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-monitoring-metrics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441691731Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver-sar.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441777043Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441862475Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/openshift-oauth-server.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.441957686Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/probes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442041168Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/service-accounts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.4421292Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-leader-election.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442245372Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-node-high.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442331164Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/flowschemas/system-nodes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442438266Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442467936Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/catch-all.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442566288Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/exempt.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44264816Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/global-default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442732002Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/leader-election.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442812063Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/node-high.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.442899465Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/openshift-control-plane-operators.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443045868Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/system.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443290753Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-high.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443437526Z cluster-scoped-resources/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/workload-low.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443442996Z cluster-scoped-resources/gateway.networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443448446Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443468466Z cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/openshift-default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443551808Z cluster-scoped-resources/imageregistry.operator.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443563378Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443580798Z cluster-scoped-resources/imageregistry.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44366181Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443697041Z cluster-scoped-resources/imageregistry.operator.openshift.io/imagepruners/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443759702Z cluster-scoped-resources/migration.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443766802Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443796473Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/console-plugin-storage-version-migration.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.443888415Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-controllerconfig-storage-version-migration.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444029167Z cluster-scoped-resources/migration.k8s.io/storageversionmigrations/machineconfiguration-machineconfigpool-storage-version-migration.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444078998Z cluster-scoped-resources/oauth.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444086019Z cluster-scoped-resources/oauth.openshift.io/oauthclients/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444127289Z cluster-scoped-resources/oauth.openshift.io/oauthclients/console.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444255382Z cluster-scoped-resources/operator.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444266442Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444305223Z cluster-scoped-resources/operator.openshift.io/clustercsidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444399815Z cluster-scoped-resources/operator.openshift.io/consoles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444442735Z cluster-scoped-resources/operator.openshift.io/consoles/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444543248Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444586768Z cluster-scoped-resources/operator.openshift.io/csisnapshotcontrollers/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44465886Z cluster-scoped-resources/operator.openshift.io/dnses/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444700791Z cluster-scoped-resources/operator.openshift.io/dnses/default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444775812Z cluster-scoped-resources/operator.openshift.io/insightsoperators/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444805153Z cluster-scoped-resources/operator.openshift.io/insightsoperators/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444931205Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.444966826Z cluster-scoped-resources/operator.openshift.io/kubeapiservers/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445037287Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445074218Z cluster-scoped-resources/operator.openshift.io/kubecontrollermanagers/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445128869Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44517461Z cluster-scoped-resources/operator.openshift.io/kubeschedulers/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445259271Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445311963Z cluster-scoped-resources/operator.openshift.io/kubestorageversionmigrators/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445384774Z cluster-scoped-resources/operator.openshift.io/networks/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445426525Z cluster-scoped-resources/operator.openshift.io/networks/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445492066Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445529327Z cluster-scoped-resources/operator.openshift.io/openshiftapiservers/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445581648Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445637079Z cluster-scoped-resources/operator.openshift.io/openshiftcontrollermanagers/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445653969Z cluster-scoped-resources/operator.openshift.io/servicecas/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44570954Z cluster-scoped-resources/operator.openshift.io/servicecas/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445769502Z cluster-scoped-resources/operator.openshift.io/storages/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445805872Z cluster-scoped-resources/operator.openshift.io/storages/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445891444Z cluster-scoped-resources/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445899424Z cluster-scoped-resources/operators.coreos.com/olmconfigs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445945825Z cluster-scoped-resources/operators.coreos.com/olmconfigs/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.445976956Z cluster-scoped-resources/operators.coreos.com/operators/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446022827Z cluster-scoped-resources/operators.coreos.com/operators/authorino-operator.kuadrant-system.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446125209Z cluster-scoped-resources/operators.coreos.com/operators/dns-operator.kuadrant-system.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446238671Z cluster-scoped-resources/operators.coreos.com/operators/leader-worker-set.openshift-lws-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446418805Z cluster-scoped-resources/operators.coreos.com/operators/limitador-operator.kuadrant-system.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446519107Z cluster-scoped-resources/operators.coreos.com/operators/openshift-cert-manager-operator.cert-manager-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446637449Z cluster-scoped-resources/operators.coreos.com/operators/openshift-custom-metrics-autoscaler-operator.openshift-keda.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446747481Z cluster-scoped-resources/operators.coreos.com/operators/rhcl-operator.kuadrant-system.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446837293Z cluster-scoped-resources/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446857213Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.446931205Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/aws-ebs-csi-driver-operator-clusterrolebinding.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447012336Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cloud-network-config-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447096358Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cluster-node-tuning-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447267651Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/cluster-storage-operator-role.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447388944Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/csi-snapshot-controller-operator-clusterrole.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447475195Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/csi-snapshot-controller-runner-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447562687Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/metrics-daemon-sa-rolebinding.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447653309Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-admission-controller-webhook.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447736381Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-ancillary-tools.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447824922Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-cluster-readers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.447909514Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-group.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448001056Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-transient.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448080667Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/multus-whereabouts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448184629Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-diagnostics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448304252Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/network-node-identity.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448395174Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-image-registry-pruner.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448501446Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-iptables-alerter.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448575817Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448660359Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-identity-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448749991Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/openshift-ovn-kubernetes-node-kube-rbac-proxy.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448835282Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/registry-registry-role.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.448991305Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449026036Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/aws-ebs-csi-driver-operator-clusterrole.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449129778Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/cloud-network-config-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.4492337Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/cluster-node-tuning-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449317712Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console-extensions-reader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449470415Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449559947Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/console.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449656739Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/csi-snapshot-controller-operator-clusterrole.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.44974245Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/helm-chartrepos-viewer.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449833682Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/metrics-daemon-role.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.449920764Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-admission-controller-webhook.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450005456Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus-ancillary-tools.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450093577Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/multus.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450183529Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/net-attach-def-project.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450411433Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-diagnostics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450502105Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/network-node-identity.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450594717Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-csi-snapshot-controller-runner.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450682079Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-iptables-alerter.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450774071Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-cluster-reader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450858792Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.450957944Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-kube-rbac-proxy.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451041956Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451137828Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-editor.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.4512591Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/openshift-ovn-kubernetes-udn-viewer.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451355502Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/project-helm-chartrepository-editor.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451442484Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-admin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451538776Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-basic-user.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451626398Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-storage-admin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451714159Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:openshift:aggregate-snapshots-to-view.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451806901Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/system:registry.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451895103Z cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/whereabouts-cni.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451939734Z cluster-scoped-resources/samples.operator.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451946614Z cluster-scoped-resources/samples.operator.openshift.io/configs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.451993625Z cluster-scoped-resources/samples.operator.openshift.io/configs/cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452054126Z cluster-scoped-resources/snapshot.storage.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452063376Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452108957Z cluster-scoped-resources/snapshot.storage.k8s.io/volumesnapshotclasses/csi-aws-vsc.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452185699Z cluster-scoped-resources/storage.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452207409Z cluster-scoped-resources/storage.k8s.io/csidrivers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.45226587Z cluster-scoped-resources/storage.k8s.io/csidrivers/ebs.csi.aws.com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452357242Z cluster-scoped-resources/storage.k8s.io/csinodes/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452400853Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-133-160.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452492395Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-137-140.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452574547Z cluster-scoped-resources/storage.k8s.io/csinodes/ip-10-0-138-196.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452640348Z cluster-scoped-resources/storage.k8s.io/storageclasses/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452690739Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp2-csi.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452857902Z cluster-scoped-resources/storage.k8s.io/storageclasses/gp3-csi.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452914343Z host_service_logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.452960844Z host_service_logs/masters/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453021585Z host_service_logs/masters/NetworkManager_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453075306Z host_service_logs/masters/crio_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453146908Z host_service_logs/masters/kubelet_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453239149Z host_service_logs/masters/machine-config-daemon-firstboot_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453370102Z host_service_logs/masters/machine-config-daemon-host_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453483274Z host_service_logs/masters/openvswitch_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453553396Z host_service_logs/masters/ostree-finalize-staged_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453627627Z host_service_logs/masters/ovs-configuration_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453701969Z host_service_logs/masters/ovs-vswitchd_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.45377125Z host_service_logs/masters/ovsdb-server_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453845401Z host_service_logs/masters/rpm-ostreed_service.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453879522Z ingress_controllers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453888462Z ingress_controllers/default/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453892622Z ingress_controllers/default/router-default-5f6bfd579b-gk6x5/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.453946963Z ingress_controllers/default/router-default-5f6bfd579b-gk6x5/haproxy.config [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.454047205Z insights-data/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.454056686Z istio/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.454112087Z istio/aggregated-discovery-api.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.45424934Z istio/aggregated-discovery-apis.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.454711559Z istio/event-filter.html [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.454857052Z istio/timestamp [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.454985774Z istio/cluster-scoped-resources/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.454992814Z istio/cluster-scoped-resources/admissionregistration.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.455005375Z istio/cluster-scoped-resources/admissionregistration.k8s.io/mutatingwebhookconfigurations/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.455050945Z istio/cluster-scoped-resources/admissionregistration.k8s.io/mutatingwebhookconfigurations/istio-sidecar-injector-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.455131337Z istio/cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.455180588Z istio/cluster-scoped-resources/admissionregistration.k8s.io/validatingwebhookconfigurations/istio-validator-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.45527595Z istio/cluster-scoped-resources/apiextensions.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.455339561Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.455391472Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/authorizationpolicies.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.455580456Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/destinationrules.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.456401662Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/envoyfilters.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.456607436Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gatewayclasses.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.456766039Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.457160047Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/gateways.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.457398751Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/grpcroutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.457806429Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/httproutes.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.458820489Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.458964162Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/inferencepools.inference.networking.x-k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.459095295Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/peerauthentications.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.459265798Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/proxyconfigs.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.4593642Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/referencegrants.gateway.networking.k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.459483732Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/requestauthentications.security.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.459631975Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/serviceentries.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.459828069Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/sidecars.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.460090644Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/telemetries.telemetry.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.460296398Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/virtualservices.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.460714806Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/wasmplugins.extensions.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.460839159Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadentries.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.460967321Z istio/cluster-scoped-resources/apiextensions.k8s.io/customresourcedefinitions/workloadgroups.networking.istio.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461152715Z istio/cluster-scoped-resources/gateway.networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461193806Z istio/cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461212246Z istio/cluster-scoped-resources/gateway.networking.k8s.io/gatewayclasses/openshift-default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461256877Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461271307Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461312198Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istio-reader-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461391449Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istiod-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461567873Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterrolebindings/istiod-gateway-controller-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461596384Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461652745Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istio-reader-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461746346Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istiod-clusterrole-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46190595Z istio/cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/istiod-gateway-controller-openshift-gateway-openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46194881Z istio/namespaces/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.461986961Z istio/namespaces/openshift-ingress/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.462044732Z istio/namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.462124044Z istio/namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.462183715Z istio/namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.462302267Z istio/namespaces/openshift-ingress/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.462341768Z istio/namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46244901Z istio/namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.462695805Z istio/namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.462896379Z istio/namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46294129Z istio/namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46297046Z istio/namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.463047482Z istio/namespaces/openshift-ingress/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.463085462Z istio/namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.463191625Z istio/namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.463261536Z istio/namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.463308357Z istio/namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.463418339Z istio/namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46348939Z istio/namespaces/openshift-ingress/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.463530221Z istio/namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46398069Z istio/namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.464081282Z istio/namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.464354687Z istio/namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.464448009Z istio/namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.464688904Z istio/namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.464818006Z istio/namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465060801Z istio/namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465148743Z istio/namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465194934Z istio/namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465308386Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465325856Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465347577Z istio/namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/openshift-ai-inference.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465430038Z istio/namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465446269Z istio/namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465545501Z istio/namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465586522Z istio/namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465678993Z istio/namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465755545Z istio/namespaces/openshift-ingress/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465802676Z istio/namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465855817Z istio/namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465920008Z istio/namespaces/openshift-ingress/networking.istio.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465929788Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.465980279Z istio/namespaces/openshift-ingress/networking.istio.io/envoyfilters/openshift-ai-inference-authn-ssl.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.46603339Z istio/namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466051571Z istio/namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466133762Z istio/namespaces/openshift-ingress/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466145392Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466157443Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/istiod-openshift-gateway-94bb8fbfd-zltzb.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466422528Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466443018Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466453938Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.466491719Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.472073398Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.47214553Z istio/namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.47215899Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.472241261Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.472377834Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.472392734Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.472405765Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.472428015Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473047577Z istio/namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473149599Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473215911Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router-default-5f6bfd579b-gk6x5.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473372214Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473387264Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473397144Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473437035Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473570567Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473629559Z istio/namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473679509Z istio/namespaces/openshift-ingress/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.47372202Z istio/namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473766321Z istio/namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473816562Z istio/namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473860643Z machine_config_ondisk/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473868123Z machine_config_termination_logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473874733Z monitoring/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473878603Z monitoring/alertmanager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.473936834Z monitoring/alertmanager/status.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.474028606Z monitoring/alertmanager/status.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.474086977Z monitoring/prometheus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.474124648Z monitoring/prometheus/alertmanagers.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.474253761Z monitoring/prometheus/alertmanagers.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.474313992Z monitoring/prometheus/rules.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.475000655Z monitoring/prometheus/rules.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.475048066Z monitoring/prometheus/prometheus-k8s-0/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.475142298Z monitoring/prometheus/prometheus-k8s-0/active-targets.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476096017Z monitoring/prometheus/prometheus-k8s-0/active-targets.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476117717Z monitoring/prometheus/prometheus-k8s-0/status/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476184249Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476341801Z monitoring/prometheus/prometheus-k8s-0/status/runtimeinfo.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476399763Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476492544Z monitoring/prometheus/prometheus-k8s-0/status/tsdb.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476532245Z monitoring/prometheus/status/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.476569156Z monitoring/prometheus/status/config.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477053786Z monitoring/prometheus/status/config.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477117797Z monitoring/prometheus/status/flags.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477250369Z monitoring/prometheus/status/flags.stderr [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477507164Z namespaces/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477515704Z namespaces/cert-manager-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477532715Z namespaces/cert-manager-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477542815Z namespaces/cert-manager-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477603846Z namespaces/cert-manager-operator/coordination.k8s.io/leases/cert-manager-operator-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477694788Z namespaces/cert-manager-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477714928Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477764029Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.477994114Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.478252289Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.478446403Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.478667287Z namespaces/cert-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.478892341Z namespaces/cert-manager-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.478940212Z namespaces/cert-manager-operator/operators.coreos.com/installplans/install-9fxqt.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479101385Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479144646Z namespaces/cert-manager-operator/operators.coreos.com/operatorconditions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479230648Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479277249Z namespaces/cert-manager-operator/operators.coreos.com/operatorgroups/openshift-cert-manager-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.47934631Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479386681Z namespaces/cert-manager-operator/operators.coreos.com/subscriptions/openshift-cert-manager-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479451242Z namespaces/cert-manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479458972Z namespaces/cert-manager/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479465893Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479512534Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479711747Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.479897891Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.480038864Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.480291849Z namespaces/cert-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.480732267Z namespaces/default/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.480781408Z namespaces/default/default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.480846509Z namespaces/default/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48089203Z namespaces/default/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.480963942Z namespaces/default/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481012163Z namespaces/default/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481094214Z namespaces/default/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481193826Z namespaces/default/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481303558Z namespaces/default/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481361079Z namespaces/default/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48139305Z namespaces/default/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481473682Z namespaces/default/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481514573Z namespaces/default/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481593594Z namespaces/default/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481654865Z namespaces/default/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481689056Z namespaces/default/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.481781378Z namespaces/default/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48187366Z namespaces/default/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48191042Z namespaces/default/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482024783Z namespaces/default/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482112744Z namespaces/default/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482374939Z namespaces/default/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482467711Z namespaces/default/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482548473Z namespaces/default/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482677825Z namespaces/default/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482848699Z namespaces/default/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48291765Z namespaces/default/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.482967121Z namespaces/default/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483030132Z namespaces/default/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483073583Z namespaces/default/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483120824Z namespaces/default/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483149145Z namespaces/default/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483359159Z namespaces/default/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48342957Z namespaces/default/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483471791Z namespaces/default/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483521562Z namespaces/default/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483586793Z namespaces/default/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483626094Z namespaces/default/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483657684Z namespaces/default/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483668375Z namespaces/default/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483712696Z namespaces/default/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.483859638Z namespaces/default/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484039952Z namespaces/default/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484199835Z namespaces/default/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48443621Z namespaces/default/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484635743Z namespaces/default/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484656094Z namespaces/default/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484734365Z namespaces/default/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484782026Z namespaces/default/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484830557Z namespaces/kserve-ci-e2e-test/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484840467Z namespaces/kserve-ci-e2e-test/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484853288Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.484899789Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.485065562Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.485287266Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.485428039Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.485639533Z namespaces/kserve-ci-e2e-test/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.485869148Z namespaces/kserve/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.485912939Z namespaces/kserve/kserve.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48600254Z namespaces/kserve/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486045661Z namespaces/kserve/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486115122Z namespaces/kserve/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486136113Z namespaces/kserve/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486272315Z namespaces/kserve/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48652352Z namespaces/kserve/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486701214Z namespaces/kserve/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486746105Z namespaces/kserve/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486785856Z namespaces/kserve/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486844137Z namespaces/kserve/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486894658Z namespaces/kserve/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.486981969Z namespaces/kserve/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487077901Z namespaces/kserve/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487121432Z namespaces/kserve/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487245405Z namespaces/kserve/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487305726Z namespaces/kserve/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487329526Z namespaces/kserve/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487365507Z namespaces/kserve/coordination.k8s.io/leases/kserve-controller-manager-leader-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487454689Z namespaces/kserve/coordination.k8s.io/leases/llminferenceservice-kserve-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48754253Z namespaces/kserve/coordination.k8s.io/leases/odh-model-controller.opendatahub.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487646832Z namespaces/kserve/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487675533Z namespaces/kserve/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.487934758Z namespaces/kserve/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48804756Z namespaces/kserve/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.488388467Z namespaces/kserve/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.488470518Z namespaces/kserve/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.488741984Z namespaces/kserve/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.488869376Z namespaces/kserve/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489127261Z namespaces/kserve/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489258064Z namespaces/kserve/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489310615Z namespaces/kserve/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489422177Z namespaces/kserve/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489452378Z namespaces/kserve/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489535979Z namespaces/kserve/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.48958074Z namespaces/kserve/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489673522Z namespaces/kserve/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489763214Z namespaces/kserve/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489793114Z namespaces/kserve/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489849255Z namespaces/kserve/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489919397Z namespaces/kserve/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.489965668Z namespaces/kserve/monitoring.coreos.com/servicemonitors/model-serving-api-metrics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.49006368Z namespaces/kserve/monitoring.coreos.com/servicemonitors/odh-model-controller-metrics-monitor.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490123101Z namespaces/kserve/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490190612Z namespaces/kserve/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490258993Z namespaces/kserve/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490271994Z namespaces/kserve/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490343105Z namespaces/kserve/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490544739Z namespaces/kserve/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490836245Z namespaces/kserve/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.490977437Z namespaces/kserve/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.491189511Z namespaces/kserve/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.491431146Z namespaces/kserve/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.491438496Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.491494288Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/kserve-controller-manager-645f4f9567-g46rm.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.491591609Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.49161081Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.491616Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.491668621Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492000107Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492033238Z namespaces/kserve/pods/kserve-controller-manager-645f4f9567-g46rm/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492047918Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492053199Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/llmisvc-controller-manager-687957b4c5-s2t9c.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492058128Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492065789Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492070429Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.492100499Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.50186954Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.501937641Z namespaces/kserve/pods/llmisvc-controller-manager-687957b4c5-s2t9c/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.501991543Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502027153Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/model-serving-api-fd65d7d6-rv7mc.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502137325Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/server/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502146346Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/server/server/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502154136Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/server/server/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502237387Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/server/server/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502317859Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/server/server/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502408781Z namespaces/kserve/pods/model-serving-api-fd65d7d6-rv7mc/server/server/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502457842Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502511833Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/odh-model-controller-84b54b7f97-d7pgn.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502586104Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502593914Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502600754Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.502657286Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523677216Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523725847Z namespaces/kserve/pods/odh-model-controller-84b54b7f97-d7pgn/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523754118Z namespaces/kserve/pods/s3-init-2bmzn/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523802029Z namespaces/kserve/pods/s3-init-2bmzn/s3-init-2bmzn.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523920031Z namespaces/kserve/pods/s3-init-2bmzn/download-hf-model/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523929401Z namespaces/kserve/pods/s3-init-2bmzn/download-hf-model/download-hf-model/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523933761Z namespaces/kserve/pods/s3-init-2bmzn/download-hf-model/download-hf-model/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.523969662Z namespaces/kserve/pods/s3-init-2bmzn/download-hf-model/download-hf-model/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524063664Z namespaces/kserve/pods/s3-init-2bmzn/download-hf-model/download-hf-model/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524139985Z namespaces/kserve/pods/s3-init-2bmzn/download-hf-model/download-hf-model/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524158656Z namespaces/kserve/pods/s3-init-2bmzn/s3-init/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524179176Z namespaces/kserve/pods/s3-init-2bmzn/s3-init/s3-init/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524187356Z namespaces/kserve/pods/s3-init-2bmzn/s3-init/s3-init/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524231977Z namespaces/kserve/pods/s3-init-2bmzn/s3-init/s3-init/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524509033Z namespaces/kserve/pods/s3-init-2bmzn/s3-init/s3-init/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524573264Z namespaces/kserve/pods/s3-init-2bmzn/s3-init/s3-init/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524612475Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524650095Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/seaweedfs-64568bcd49-bn4s5.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524731557Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/seaweedfs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524739217Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/seaweedfs/seaweedfs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524745957Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/seaweedfs/seaweedfs/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.524799328Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/seaweedfs/seaweedfs/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525014222Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/seaweedfs/seaweedfs/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525080004Z namespaces/kserve/pods/seaweedfs-64568bcd49-bn4s5/seaweedfs/seaweedfs/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525121054Z namespaces/kserve/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525149065Z namespaces/kserve/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525252617Z namespaces/kserve/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525294028Z namespaces/kserve/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.52538943Z namespaces/kuadrant-system/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525438961Z namespaces/kuadrant-system/kuadrant-system.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525512982Z namespaces/kuadrant-system/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525555513Z namespaces/kuadrant-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525622564Z namespaces/kuadrant-system/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525665115Z namespaces/kuadrant-system/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.525759067Z namespaces/kuadrant-system/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526068253Z namespaces/kuadrant-system/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526351249Z namespaces/kuadrant-system/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.52642756Z namespaces/kuadrant-system/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526472621Z namespaces/kuadrant-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526532012Z namespaces/kuadrant-system/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526584033Z namespaces/kuadrant-system/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526664195Z namespaces/kuadrant-system/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526706405Z namespaces/kuadrant-system/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526744346Z namespaces/kuadrant-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526824788Z namespaces/kuadrant-system/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526870819Z namespaces/kuadrant-system/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.526899129Z namespaces/kuadrant-system/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.52694404Z namespaces/kuadrant-system/coordination.k8s.io/leases/3745a16e.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.527035702Z namespaces/kuadrant-system/coordination.k8s.io/leases/a3f98d6c.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.527125164Z namespaces/kuadrant-system/coordination.k8s.io/leases/aac3a15d.authorino.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.527229066Z namespaces/kuadrant-system/coordination.k8s.io/leases/f139389e.kuadrant.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.527261586Z namespaces/kuadrant-system/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.527318257Z namespaces/kuadrant-system/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.52744839Z namespaces/kuadrant-system/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.527565172Z namespaces/kuadrant-system/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.52796159Z namespaces/kuadrant-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.528038071Z namespaces/kuadrant-system/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.528512041Z namespaces/kuadrant-system/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.528647763Z namespaces/kuadrant-system/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.528885648Z namespaces/kuadrant-system/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.52900289Z namespaces/kuadrant-system/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529030791Z namespaces/kuadrant-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529147103Z namespaces/kuadrant-system/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529206344Z namespaces/kuadrant-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529252245Z namespaces/kuadrant-system/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529303666Z namespaces/kuadrant-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529385408Z namespaces/kuadrant-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529460889Z namespaces/kuadrant-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529559551Z namespaces/kuadrant-system/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529590332Z namespaces/kuadrant-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529646923Z namespaces/kuadrant-system/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529678123Z namespaces/kuadrant-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529753315Z namespaces/kuadrant-system/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529762765Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.529816966Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.53000771Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.530263485Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.530434958Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.530656902Z namespaces/kuadrant-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.530870047Z namespaces/kuadrant-system/operators.coreos.com/installplans/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.530900057Z namespaces/kuadrant-system/operators.coreos.com/installplans/install-xzwn4.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531250164Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531304535Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531411687Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531505289Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531582121Z namespaces/kuadrant-system/operators.coreos.com/operatorconditions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531625581Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531657582Z namespaces/kuadrant-system/operators.coreos.com/operatorgroups/kuadrant.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531721933Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531759654Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/authorino-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.531857206Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/dns-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532016299Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/limitador-operator-stable-redhat-operators-openshift-marketplace.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532112381Z namespaces/kuadrant-system/operators.coreos.com/subscriptions/rhcl-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532228063Z namespaces/kuadrant-system/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532244584Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532266394Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino-7c77957575-mjgp9.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532365336Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532379206Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532390496Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.532432277Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536281743Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536349714Z namespaces/kuadrant-system/pods/authorino-7c77957575-mjgp9/authorino/authorino/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536392425Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536422515Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/authorino-operator-6d85f6564-kxw42.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536547928Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536556268Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536568288Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536620249Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536798373Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536865194Z namespaces/kuadrant-system/pods/authorino-operator-6d85f6564-kxw42/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536909615Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.536973166Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/dns-operator-controller-manager-65b49595d7-ld9rx.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537053818Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537066158Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537078468Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537107568Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537244591Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537308883Z namespaces/kuadrant-system/pods/dns-operator-controller-manager-65b49595d7-ld9rx/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537378524Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537404134Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin-85fd4b8b9d-7fxgf.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537492936Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537502986Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537522907Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537566237Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537642169Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537714111Z namespaces/kuadrant-system/pods/kuadrant-console-plugin-85fd4b8b9d-7fxgf/kuadrant-console-plugin/kuadrant-console-plugin/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537733021Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537787402Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537906074Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537915954Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537922504Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.537931645Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544496913Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544572284Z namespaces/kuadrant-system/pods/kuadrant-operator-controller-manager-bb7ddb7d7-r96fg/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544593865Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544645986Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador-limitador-69574b596d-q97v8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544730817Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544741898Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544747028Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544759148Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.54486245Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544924181Z namespaces/kuadrant-system/pods/limitador-limitador-69574b596d-q97v8/limitador/limitador/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.544978762Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545011183Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/limitador-operator-controller-manager-6f9f468797-rvhvm.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545108215Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545117525Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545132425Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545157086Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545322859Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.54539119Z namespaces/kuadrant-system/pods/limitador-operator-controller-manager-6f9f468797-rvhvm/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545441221Z namespaces/kuadrant-system/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545461592Z namespaces/kuadrant-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545526533Z namespaces/kuadrant-system/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545579134Z namespaces/kuadrant-system/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545641715Z namespaces/kube-node-lease/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545650015Z namespaces/kube-node-lease/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545654186Z namespaces/kube-node-lease/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545696356Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-133-160.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545786648Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-137-140.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.54586454Z namespaces/kube-node-lease/coordination.k8s.io/leases/ip-10-0-138-196.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5459126Z namespaces/kube-node-lease/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545935011Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.545988352Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.546187326Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.54642411Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.546571533Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.546857579Z namespaces/kube-node-lease/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547084803Z namespaces/kube-public/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547095614Z namespaces/kube-public/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547106094Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547182595Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547336879Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547521602Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547668575Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.547873409Z namespaces/kube-public/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548107554Z namespaces/kube-system/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548152054Z namespaces/kube-system/kube-system.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548309458Z namespaces/kube-system/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548353648Z namespaces/kube-system/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.54841997Z namespaces/kube-system/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548465541Z namespaces/kube-system/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548588653Z namespaces/kube-system/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548666384Z namespaces/kube-system/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548742826Z namespaces/kube-system/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548802027Z namespaces/kube-system/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548832758Z namespaces/kube-system/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.548907139Z namespaces/kube-system/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.54893678Z namespaces/kube-system/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549027412Z namespaces/kube-system/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549057742Z namespaces/kube-system/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549107493Z namespaces/kube-system/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549208225Z namespaces/kube-system/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549262446Z namespaces/kube-system/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549274806Z namespaces/kube-system/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549303767Z namespaces/kube-system/coordination.k8s.io/leases/apiserver-j4soh2paurc2ukexoptubvbgmu.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549389228Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-cainjector-leader-election.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549497701Z namespaces/kube-system/coordination.k8s.io/leases/cert-manager-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549580712Z namespaces/kube-system/coordination.k8s.io/leases/kube-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549733275Z namespaces/kube-system/coordination.k8s.io/leases/kube-scheduler.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549800516Z namespaces/kube-system/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.549846697Z namespaces/kube-system/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.54999286Z namespaces/kube-system/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.550074972Z namespaces/kube-system/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.550364027Z namespaces/kube-system/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.550453019Z namespaces/kube-system/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.550736935Z namespaces/kube-system/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.550862507Z namespaces/kube-system/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551230975Z namespaces/kube-system/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551292446Z namespaces/kube-system/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551374297Z namespaces/kube-system/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551426508Z namespaces/kube-system/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551462479Z namespaces/kube-system/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55153616Z namespaces/kube-system/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551586081Z namespaces/kube-system/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551675083Z namespaces/kube-system/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551758055Z namespaces/kube-system/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551815276Z namespaces/kube-system/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551867787Z namespaces/kube-system/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551916948Z namespaces/kube-system/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.551954218Z namespaces/kube-system/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55202161Z namespaces/kube-system/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55202869Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.552065331Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.552244294Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.552434738Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.552575361Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.552769854Z namespaces/kube-system/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.552981619Z namespaces/kube-system/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.552991959Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55304986Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/global-pull-secret-syncer-2tscg.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553128752Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/global-pull-secret-syncer/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553136152Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553140552Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553195923Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55355019Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553617641Z namespaces/kube-system/pods/global-pull-secret-syncer-2tscg/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553678962Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553723723Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/global-pull-secret-syncer-n46hj.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553794325Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/global-pull-secret-syncer/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553807205Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553813095Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.553834145Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554150652Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554234023Z namespaces/kube-system/pods/global-pull-secret-syncer-n46hj/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554258974Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554307375Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/global-pull-secret-syncer-tjzjf.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554377846Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/global-pull-secret-syncer/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554396856Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/global-pull-secret-syncer/global-pull-secret-syncer/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554401456Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/global-pull-secret-syncer/global-pull-secret-syncer/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554462728Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/global-pull-secret-syncer/global-pull-secret-syncer/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554782764Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554860495Z namespaces/kube-system/pods/global-pull-secret-syncer-tjzjf/global-pull-secret-syncer/global-pull-secret-syncer/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554877186Z namespaces/kube-system/pods/konnectivity-agent-42cdx/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.554941327Z namespaces/kube-system/pods/konnectivity-agent-42cdx/konnectivity-agent-42cdx.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.555008778Z namespaces/kube-system/pods/konnectivity-agent-42cdx/konnectivity-agent/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.555020718Z namespaces/kube-system/pods/konnectivity-agent-42cdx/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.555036589Z namespaces/kube-system/pods/konnectivity-agent-42cdx/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.555072719Z namespaces/kube-system/pods/konnectivity-agent-42cdx/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.556079039Z namespaces/kube-system/pods/konnectivity-agent-42cdx/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55614274Z namespaces/kube-system/pods/konnectivity-agent-42cdx/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.556213672Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.556257393Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/konnectivity-agent-dgsrm.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.556343814Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/konnectivity-agent/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.556351435Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.556355575Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.556406496Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.557537998Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.557604279Z namespaces/kube-system/pods/konnectivity-agent-dgsrm/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55766699Z namespaces/kube-system/pods/konnectivity-agent-hchgr/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.557708341Z namespaces/kube-system/pods/konnectivity-agent-hchgr/konnectivity-agent-hchgr.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.557787372Z namespaces/kube-system/pods/konnectivity-agent-hchgr/konnectivity-agent/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.557797293Z namespaces/kube-system/pods/konnectivity-agent-hchgr/konnectivity-agent/konnectivity-agent/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.557807043Z namespaces/kube-system/pods/konnectivity-agent-hchgr/konnectivity-agent/konnectivity-agent/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.557827393Z namespaces/kube-system/pods/konnectivity-agent-hchgr/konnectivity-agent/konnectivity-agent/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.558890394Z namespaces/kube-system/pods/konnectivity-agent-hchgr/konnectivity-agent/konnectivity-agent/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.558955175Z namespaces/kube-system/pods/konnectivity-agent-hchgr/konnectivity-agent/konnectivity-agent/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.558986806Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559026827Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559116858Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/haproxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559125159Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559129359Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.55918102Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559458755Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559523797Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559582148Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.559601878Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560031027Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/haproxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560041947Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560046997Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560084308Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560416814Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560485115Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-137-140.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560535716Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560566877Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560650179Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/haproxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560657729Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/haproxy/haproxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560664139Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/haproxy/haproxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560705929Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/haproxy/haproxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.560942894Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/haproxy/haproxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561014275Z namespaces/kube-system/pods/kube-apiserver-proxy-ip-10-0-138-196.ec2.internal/haproxy/haproxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561048706Z namespaces/kube-system/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561086877Z namespaces/kube-system/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561126638Z namespaces/kube-system/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561133998Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561179759Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/csi-snapshot-controller-operator-authentication-reader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561292371Z namespaces/kube-system/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561339942Z namespaces/kube-system/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561383473Z namespaces/kube-system/rbac.authorization.k8s.io/roles/extension-apiserver-authentication-reader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561443684Z namespaces/kube-system/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561484375Z namespaces/kube-system/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561539636Z namespaces/observability/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561548276Z namespaces/observability/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561559416Z namespaces/observability/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561604427Z namespaces/observability/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561797961Z namespaces/observability/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.561988865Z namespaces/observability/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.562128897Z namespaces/observability/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.562346392Z namespaces/observability/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.562545016Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.562553306Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.562565236Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.562608837Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5627716Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.562968424Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563105336Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563330791Z namespaces/open-cluster-management-32f21fc7-bfe9-438c-880b-7e8aaa8ec/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563525585Z namespaces/open-cluster-management-agent-addon/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563533515Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563537795Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563558885Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/cluster-proxy.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563663027Z namespaces/open-cluster-management-agent-addon/coordination.k8s.io/leases/work-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563703398Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563715018Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563770969Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.563926872Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564107646Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564262029Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564462523Z namespaces/open-cluster-management-agent-addon/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564688767Z namespaces/openshift-apiserver-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564734798Z namespaces/openshift-apiserver-operator/openshift-apiserver-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.56481052Z namespaces/openshift-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564846701Z namespaces/openshift-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564916352Z namespaces/openshift-apiserver-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.564959243Z namespaces/openshift-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565040934Z namespaces/openshift-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565114256Z namespaces/openshift-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565259198Z namespaces/openshift-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565286089Z namespaces/openshift-apiserver-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5653448Z namespaces/openshift-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565394941Z namespaces/openshift-apiserver-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565444572Z namespaces/openshift-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565517804Z namespaces/openshift-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565566484Z namespaces/openshift-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565604235Z namespaces/openshift-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565680377Z namespaces/openshift-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565739008Z namespaces/openshift-apiserver-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.565767068Z namespaces/openshift-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.566732097Z namespaces/openshift-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.566815439Z namespaces/openshift-apiserver-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.56689577Z namespaces/openshift-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.566980512Z namespaces/openshift-apiserver-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567064654Z namespaces/openshift-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567187606Z namespaces/openshift-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.56738931Z namespaces/openshift-apiserver-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567458181Z namespaces/openshift-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567506742Z namespaces/openshift-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567560594Z namespaces/openshift-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567579854Z namespaces/openshift-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567656005Z namespaces/openshift-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567699166Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567773808Z namespaces/openshift-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.567856359Z namespaces/openshift-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568041993Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568081284Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568205666Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568267777Z namespaces/openshift-apiserver-operator/monitoring.coreos.com/servicemonitors/openshift-apiserver-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568373619Z namespaces/openshift-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.56838805Z namespaces/openshift-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568496972Z namespaces/openshift-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568533382Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568565283Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.568694396Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.56893453Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569096173Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569324318Z namespaces/openshift-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569544822Z namespaces/openshift-apiserver-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569588113Z namespaces/openshift-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569627934Z namespaces/openshift-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569672655Z namespaces/openshift-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569781827Z namespaces/openshift-apiserver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569827338Z namespaces/openshift-apiserver/openshift-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569897869Z namespaces/openshift-apiserver/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5699397Z namespaces/openshift-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.569997001Z namespaces/openshift-apiserver/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570030842Z namespaces/openshift-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570117253Z namespaces/openshift-apiserver/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570227626Z namespaces/openshift-apiserver/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570321037Z namespaces/openshift-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570392239Z namespaces/openshift-apiserver/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57043535Z namespaces/openshift-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570498651Z namespaces/openshift-apiserver/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570534232Z namespaces/openshift-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570624363Z namespaces/openshift-apiserver/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570672654Z namespaces/openshift-apiserver/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570721475Z namespaces/openshift-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570806287Z namespaces/openshift-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570883228Z namespaces/openshift-apiserver/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.570928999Z namespaces/openshift-apiserver/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571033721Z namespaces/openshift-apiserver/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571115703Z namespaces/openshift-apiserver/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571217175Z namespaces/openshift-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571300856Z namespaces/openshift-apiserver/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571386868Z namespaces/openshift-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571511121Z namespaces/openshift-apiserver/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571666054Z namespaces/openshift-apiserver/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571715555Z namespaces/openshift-apiserver/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571756565Z namespaces/openshift-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571812506Z namespaces/openshift-apiserver/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571861138Z namespaces/openshift-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.571920369Z namespaces/openshift-apiserver/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57197143Z namespaces/openshift-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572049801Z namespaces/openshift-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572138713Z namespaces/openshift-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572211444Z namespaces/openshift-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572255335Z namespaces/openshift-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572318087Z namespaces/openshift-apiserver/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572363397Z namespaces/openshift-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572411498Z namespaces/openshift-apiserver/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572429179Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572475279Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572629782Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572809996Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.572967309Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573178003Z namespaces/openshift-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573366737Z namespaces/openshift-apiserver/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573413928Z namespaces/openshift-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573472449Z namespaces/openshift-apiserver/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57351586Z namespaces/openshift-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573557841Z namespaces/openshift-authentication-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573565791Z namespaces/openshift-authentication-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573573221Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573622172Z namespaces/openshift-authentication-operator/monitoring.coreos.com/servicemonitors/authentication-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573684053Z namespaces/openshift-authentication-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573698883Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.573756064Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.574111142Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.574365947Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57452299Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.574734494Z namespaces/openshift-authentication-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.574907117Z namespaces/openshift-authentication/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.574915757Z namespaces/openshift-authentication/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.574932378Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.574985648Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575138271Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575343746Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575489428Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575697813Z namespaces/openshift-authentication/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575881956Z namespaces/openshift-cloud-controller-manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575889406Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575902086Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.575942217Z namespaces/openshift-cloud-controller-manager/coordination.k8s.io/leases/cloud-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.576007819Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.576018529Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57607018Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.576248833Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.576429527Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.576566119Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.576777574Z namespaces/openshift-cloud-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577058349Z namespaces/openshift-cloud-credential-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577069379Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577073569Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57711304Z namespaces/openshift-cloud-credential-operator/coordination.k8s.io/leases/cloud-credential-operator-leader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577185431Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577209152Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577231792Z namespaces/openshift-cloud-credential-operator/monitoring.coreos.com/servicemonitors/cloud-credential-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577321344Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577330774Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577376915Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577531378Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577705872Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.577842284Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578066619Z namespaces/openshift-cloud-credential-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578304643Z namespaces/openshift-cloud-network-config-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578329794Z namespaces/openshift-cloud-network-config-controller/openshift-cloud-network-config-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578408725Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578445136Z namespaces/openshift-cloud-network-config-controller/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578518637Z namespaces/openshift-cloud-network-config-controller/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578547348Z namespaces/openshift-cloud-network-config-controller/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57864187Z namespaces/openshift-cloud-network-config-controller/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578723352Z namespaces/openshift-cloud-network-config-controller/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578801803Z namespaces/openshift-cloud-network-config-controller/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578857584Z namespaces/openshift-cloud-network-config-controller/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578872435Z namespaces/openshift-cloud-network-config-controller/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578929986Z namespaces/openshift-cloud-network-config-controller/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.578968286Z namespaces/openshift-cloud-network-config-controller/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579057828Z namespaces/openshift-cloud-network-config-controller/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579095649Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57914296Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579249512Z namespaces/openshift-cloud-network-config-controller/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579299603Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579309873Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579353614Z namespaces/openshift-cloud-network-config-controller/coordination.k8s.io/leases/cloud-network-config-controller-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579444966Z namespaces/openshift-cloud-network-config-controller/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579490116Z namespaces/openshift-cloud-network-config-controller/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579595278Z namespaces/openshift-cloud-network-config-controller/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.57967721Z namespaces/openshift-cloud-network-config-controller/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579758722Z namespaces/openshift-cloud-network-config-controller/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579835823Z namespaces/openshift-cloud-network-config-controller/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.579919465Z namespaces/openshift-cloud-network-config-controller/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580070798Z namespaces/openshift-cloud-network-config-controller/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58019298Z namespaces/openshift-cloud-network-config-controller/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580244451Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580294482Z namespaces/openshift-cloud-network-config-controller/core/serviceaccounts/cloud-network-config-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580337383Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580392254Z namespaces/openshift-cloud-network-config-controller/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580420565Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580475546Z namespaces/openshift-cloud-network-config-controller/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580556717Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580606118Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5806906Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580770441Z namespaces/openshift-cloud-network-config-controller/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580817682Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580877104Z namespaces/openshift-cloud-network-config-controller/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580889274Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580943085Z namespaces/openshift-cloud-network-config-controller/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.580991646Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.581002146Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.581036857Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5811943Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.581487606Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.581649369Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.581845062Z namespaces/openshift-cloud-network-config-controller/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582033776Z namespaces/openshift-cloud-network-config-controller/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582067197Z namespaces/openshift-cloud-network-config-controller/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582119288Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582128678Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582181299Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/rolebindings/cloud-network-config-controller-rb.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5822383Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582278091Z namespaces/openshift-cloud-network-config-controller/rbac.authorization.k8s.io/roles/cloud-network-config-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582332622Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582368833Z namespaces/openshift-cloud-network-config-controller/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582475825Z namespaces/openshift-cluster-csi-drivers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582504075Z namespaces/openshift-cluster-csi-drivers/openshift-cluster-csi-drivers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582590657Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582623188Z namespaces/openshift-cluster-csi-drivers/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582693659Z namespaces/openshift-cluster-csi-drivers/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58274364Z namespaces/openshift-cluster-csi-drivers/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.582953434Z namespaces/openshift-cluster-csi-drivers/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583029886Z namespaces/openshift-cluster-csi-drivers/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583109167Z namespaces/openshift-cluster-csi-drivers/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583162208Z namespaces/openshift-cluster-csi-drivers/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583219499Z namespaces/openshift-cluster-csi-drivers/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58326443Z namespaces/openshift-cluster-csi-drivers/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583320991Z namespaces/openshift-cluster-csi-drivers/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583408413Z namespaces/openshift-cluster-csi-drivers/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583473034Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583494985Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583584627Z namespaces/openshift-cluster-csi-drivers/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583640118Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583647938Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583693279Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/ebs-csi-aws-com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58378024Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-attacher-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583864162Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-resizer-ebs-csi-aws-com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.583980844Z namespaces/openshift-cluster-csi-drivers/coordination.k8s.io/leases/external-snapshotter-leader-ebs-csi-aws-com.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.584076006Z namespaces/openshift-cluster-csi-drivers/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.584120127Z namespaces/openshift-cluster-csi-drivers/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.584248129Z namespaces/openshift-cluster-csi-drivers/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.584327441Z namespaces/openshift-cluster-csi-drivers/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.584660197Z namespaces/openshift-cluster-csi-drivers/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.584740389Z namespaces/openshift-cluster-csi-drivers/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.584950413Z namespaces/openshift-cluster-csi-drivers/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585086866Z namespaces/openshift-cluster-csi-drivers/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585228348Z namespaces/openshift-cluster-csi-drivers/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58528077Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58532295Z namespaces/openshift-cluster-csi-drivers/core/serviceaccounts/aws-ebs-csi-driver-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585376822Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585422602Z namespaces/openshift-cluster-csi-drivers/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585466853Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585510074Z namespaces/openshift-cluster-csi-drivers/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585555845Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585601666Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585679897Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585761599Z namespaces/openshift-cluster-csi-drivers/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58580936Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585851441Z namespaces/openshift-cluster-csi-drivers/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585902142Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.585946093Z namespaces/openshift-cluster-csi-drivers/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586018794Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586029254Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586085865Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586258869Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586452092Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586600085Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586794659Z namespaces/openshift-cluster-csi-drivers/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.586996353Z namespaces/openshift-cluster-csi-drivers/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587005823Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587054404Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/aws-ebs-csi-driver-node-6l2kd.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587180847Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-driver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587192857Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-driver/csi-driver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587208967Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-driver/csi-driver/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587234788Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58734962Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587420221Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587451792Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-liveness-probe/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587458012Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587464592Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587518013Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587604865Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587677567Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587723027Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-node-driver-registrar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587730788Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587741218Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587796419Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587884231Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.587957642Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-6l2kd/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588007113Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588051824Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/aws-ebs-csi-driver-node-lmftp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588140756Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-driver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588149146Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-driver/csi-driver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588155636Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-driver/csi-driver/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588236907Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588326409Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58839647Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588434521Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-liveness-probe/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588442511Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588448891Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588501532Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588596865Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588668686Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588708187Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-node-driver-registrar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588719487Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588726767Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588774538Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5888692Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588934231Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-lmftp/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.588993322Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589039523Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/aws-ebs-csi-driver-node-pgjl2.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589126125Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-driver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589139215Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-driver/csi-driver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589145265Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-driver/csi-driver/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589281728Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-driver/csi-driver/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589465411Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-driver/csi-driver/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589537083Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-driver/csi-driver/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589587104Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-liveness-probe/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589595204Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-liveness-probe/csi-liveness-probe/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589599114Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-liveness-probe/csi-liveness-probe/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589642195Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-liveness-probe/csi-liveness-probe/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589734807Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-liveness-probe/csi-liveness-probe/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589802258Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-liveness-probe/csi-liveness-probe/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589849569Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-node-driver-registrar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589861669Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-node-driver-registrar/csi-node-driver-registrar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.589869059Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-node-driver-registrar/csi-node-driver-registrar/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.58991302Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-node-driver-registrar/csi-node-driver-registrar/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590011772Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590074313Z namespaces/openshift-cluster-csi-drivers/pods/aws-ebs-csi-driver-node-pgjl2/csi-node-driver-registrar/csi-node-driver-registrar/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590127774Z namespaces/openshift-cluster-csi-drivers/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590183025Z namespaces/openshift-cluster-csi-drivers/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590248017Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590255367Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590305068Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/rolebindings/aws-ebs-csi-driver-operator-rolebinding.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590366959Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.59041986Z namespaces/openshift-cluster-csi-drivers/rbac.authorization.k8s.io/roles/aws-ebs-csi-driver-operator-role.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590470881Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590520242Z namespaces/openshift-cluster-csi-drivers/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590582943Z namespaces/openshift-cluster-machine-approver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590590003Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590593793Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590641244Z namespaces/openshift-cluster-machine-approver/coordination.k8s.io/leases/cluster-machine-approver-leader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590711556Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590721776Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590757517Z namespaces/openshift-cluster-machine-approver/monitoring.coreos.com/servicemonitors/cluster-machine-approver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590817548Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590831798Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.590877869Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.591030232Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.591231286Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.591379819Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.591588503Z namespaces/openshift-cluster-machine-approver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.591868368Z namespaces/openshift-cluster-node-tuning-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.591917979Z namespaces/openshift-cluster-node-tuning-operator/openshift-cluster-node-tuning-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592015771Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592056512Z namespaces/openshift-cluster-node-tuning-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592134803Z namespaces/openshift-cluster-node-tuning-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592176924Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592328117Z namespaces/openshift-cluster-node-tuning-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592411519Z namespaces/openshift-cluster-node-tuning-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592496821Z namespaces/openshift-cluster-node-tuning-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592546742Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592600413Z namespaces/openshift-cluster-node-tuning-operator/apps/daemonsets/tuned.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592678164Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592728245Z namespaces/openshift-cluster-node-tuning-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592774906Z namespaces/openshift-cluster-node-tuning-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592808337Z namespaces/openshift-cluster-node-tuning-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.592903238Z namespaces/openshift-cluster-node-tuning-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.59294917Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.59299515Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.593111893Z namespaces/openshift-cluster-node-tuning-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.593160454Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.593185024Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.593249845Z namespaces/openshift-cluster-node-tuning-operator/coordination.k8s.io/leases/node-tuning-operator-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.593336427Z namespaces/openshift-cluster-node-tuning-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.593388148Z namespaces/openshift-cluster-node-tuning-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.594261505Z namespaces/openshift-cluster-node-tuning-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.594412418Z namespaces/openshift-cluster-node-tuning-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.594660603Z namespaces/openshift-cluster-node-tuning-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.594738844Z namespaces/openshift-cluster-node-tuning-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.594945128Z namespaces/openshift-cluster-node-tuning-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595110162Z namespaces/openshift-cluster-node-tuning-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595275145Z namespaces/openshift-cluster-node-tuning-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595342636Z namespaces/openshift-cluster-node-tuning-operator/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595391177Z namespaces/openshift-cluster-node-tuning-operator/core/serviceaccounts/cluster-node-tuning-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595432548Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595483009Z namespaces/openshift-cluster-node-tuning-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.59554448Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595601531Z namespaces/openshift-cluster-node-tuning-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595647372Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595691303Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595779425Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.595855416Z namespaces/openshift-cluster-node-tuning-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596008849Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.59604627Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596117531Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596189373Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/prometheusrules/node-tuning-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596264674Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596311175Z namespaces/openshift-cluster-node-tuning-operator/monitoring.coreos.com/servicemonitors/node-tuning-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596372086Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596398397Z namespaces/openshift-cluster-node-tuning-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596454988Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596467678Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596537149Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596740333Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.596947628Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5970938Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597397796Z namespaces/openshift-cluster-node-tuning-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597617701Z namespaces/openshift-cluster-node-tuning-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597628311Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597704492Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/tuned-g4rfz.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597807344Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/tuned/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597816934Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/tuned/tuned/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597826335Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/tuned/tuned/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597881036Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/tuned/tuned/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.597994918Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598069179Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-g4rfz/tuned/tuned/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598222762Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598250213Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/tuned-k9hjp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598350875Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/tuned/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598358895Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/tuned/tuned/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598363115Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/tuned/tuned/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598400896Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/tuned/tuned/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598530808Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598588029Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-k9hjp/tuned/tuned/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598632701Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598673391Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/tuned-q5ws5.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598763963Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/tuned/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598773733Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/tuned/tuned/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598779603Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/tuned/tuned/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598810814Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/tuned/tuned/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.598916576Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/tuned/tuned/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599003878Z namespaces/openshift-cluster-node-tuning-operator/pods/tuned-q5ws5/tuned/tuned/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599019038Z namespaces/openshift-cluster-node-tuning-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599065669Z namespaces/openshift-cluster-node-tuning-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.5991093Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599156201Z namespaces/openshift-cluster-node-tuning-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599222622Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599242052Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599298554Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-133-160.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599382275Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-137-140.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599468427Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/profiles/ip-10-0-138-196.ec2.internal.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599536338Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599551348Z namespaces/openshift-cluster-node-tuning-operator/tuned.openshift.io/tuneds/default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.59964918Z namespaces/openshift-cluster-samples-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599699191Z namespaces/openshift-cluster-samples-operator/openshift-cluster-samples-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599961636Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.599998867Z namespaces/openshift-cluster-samples-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600063678Z namespaces/openshift-cluster-samples-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600114639Z namespaces/openshift-cluster-samples-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600227472Z namespaces/openshift-cluster-samples-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600371404Z namespaces/openshift-cluster-samples-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600486787Z namespaces/openshift-cluster-samples-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600541508Z namespaces/openshift-cluster-samples-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600590379Z namespaces/openshift-cluster-samples-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6006346Z namespaces/openshift-cluster-samples-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60068077Z namespaces/openshift-cluster-samples-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600761742Z namespaces/openshift-cluster-samples-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600822453Z namespaces/openshift-cluster-samples-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600873554Z namespaces/openshift-cluster-samples-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.600954796Z namespaces/openshift-cluster-samples-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.601039177Z namespaces/openshift-cluster-samples-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.601074298Z namespaces/openshift-cluster-samples-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.601278492Z namespaces/openshift-cluster-samples-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.601362734Z namespaces/openshift-cluster-samples-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.601492766Z namespaces/openshift-cluster-samples-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.601580488Z namespaces/openshift-cluster-samples-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60170311Z namespaces/openshift-cluster-samples-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.601929475Z namespaces/openshift-cluster-samples-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602107808Z namespaces/openshift-cluster-samples-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60218257Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602227401Z namespaces/openshift-cluster-samples-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602298762Z namespaces/openshift-cluster-samples-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602338903Z namespaces/openshift-cluster-samples-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602422995Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602469425Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602553687Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602633059Z namespaces/openshift-cluster-samples-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60268598Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60273355Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602783282Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602809742Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/prometheusrules/samples-operator-alerts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602908984Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602946945Z namespaces/openshift-cluster-samples-operator/monitoring.coreos.com/servicemonitors/cluster-samples-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.602995236Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603046287Z namespaces/openshift-cluster-samples-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603093048Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603112618Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603125868Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603301682Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603485305Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603627318Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.603866203Z namespaces/openshift-cluster-samples-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604064867Z namespaces/openshift-cluster-samples-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604078247Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604090707Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator-79b7c55f87-7jrnv.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60422371Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator-watch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60424236Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator-watch/cluster-samples-operator-watch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60425022Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604279461Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604378573Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604445374Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator-watch/cluster-samples-operator-watch/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604492845Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604522905Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator/cluster-samples-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604528686Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator/cluster-samples-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.604537296Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator/cluster-samples-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60527924Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator/cluster-samples-operator/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605342621Z namespaces/openshift-cluster-samples-operator/pods/cluster-samples-operator-79b7c55f87-7jrnv/cluster-samples-operator/cluster-samples-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605401063Z namespaces/openshift-cluster-samples-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605417443Z namespaces/openshift-cluster-samples-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605499484Z namespaces/openshift-cluster-samples-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605535285Z namespaces/openshift-cluster-samples-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605629587Z namespaces/openshift-cluster-storage-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605674448Z namespaces/openshift-cluster-storage-operator/openshift-cluster-storage-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605734839Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60577762Z namespaces/openshift-cluster-storage-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605847161Z namespaces/openshift-cluster-storage-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605890622Z namespaces/openshift-cluster-storage-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.605977144Z namespaces/openshift-cluster-storage-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606084306Z namespaces/openshift-cluster-storage-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606211989Z namespaces/openshift-cluster-storage-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60626943Z namespaces/openshift-cluster-storage-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606316451Z namespaces/openshift-cluster-storage-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606355771Z namespaces/openshift-cluster-storage-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606430823Z namespaces/openshift-cluster-storage-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606512994Z namespaces/openshift-cluster-storage-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606548625Z namespaces/openshift-cluster-storage-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606596216Z namespaces/openshift-cluster-storage-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6067805Z namespaces/openshift-cluster-storage-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60682027Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606843701Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606888812Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/data-source-validator-leader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.606976044Z namespaces/openshift-cluster-storage-operator/coordination.k8s.io/leases/snapshot-controller-leader.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.607064525Z namespaces/openshift-cluster-storage-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.607092206Z namespaces/openshift-cluster-storage-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.607241649Z namespaces/openshift-cluster-storage-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60732764Z namespaces/openshift-cluster-storage-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.607692638Z namespaces/openshift-cluster-storage-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.607772459Z namespaces/openshift-cluster-storage-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.607887081Z namespaces/openshift-cluster-storage-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608031314Z namespaces/openshift-cluster-storage-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608179987Z namespaces/openshift-cluster-storage-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608268129Z namespaces/openshift-cluster-storage-operator/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608371971Z namespaces/openshift-cluster-storage-operator/core/serviceaccounts/csi-snapshot-controller-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608437262Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608492203Z namespaces/openshift-cluster-storage-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608555114Z namespaces/openshift-cluster-storage-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608605305Z namespaces/openshift-cluster-storage-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608662937Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608722158Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608806889Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608886641Z namespaces/openshift-cluster-storage-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608947912Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.608999623Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609076074Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609129386Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/prometheusrules/prometheus.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609242438Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609275628Z namespaces/openshift-cluster-storage-operator/monitoring.coreos.com/servicemonitors/cluster-storage-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60934244Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6093703Z namespaces/openshift-cluster-storage-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609465352Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609474822Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609538023Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.609690737Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.60987233Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610009633Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610223307Z namespaces/openshift-cluster-storage-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610427281Z namespaces/openshift-cluster-storage-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610438421Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610473792Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/volume-data-source-validator-78cf4c6f6-dhvkq.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610553413Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/volume-data-source-validator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610562583Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/volume-data-source-validator/volume-data-source-validator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610567083Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/volume-data-source-validator/volume-data-source-validator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610598344Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/volume-data-source-validator/volume-data-source-validator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610705306Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/volume-data-source-validator/volume-data-source-validator/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610780188Z namespaces/openshift-cluster-storage-operator/pods/volume-data-source-validator-78cf4c6f6-dhvkq/volume-data-source-validator/volume-data-source-validator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610818388Z namespaces/openshift-cluster-storage-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610859319Z namespaces/openshift-cluster-storage-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61091336Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610925421Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.610936591Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/rolebindings/csi-snapshot-controller-operator-role.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611012802Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611059723Z namespaces/openshift-cluster-storage-operator/rbac.authorization.k8s.io/roles/csi-snapshot-controller-operator-role.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611127094Z namespaces/openshift-cluster-storage-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611176095Z namespaces/openshift-cluster-storage-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611279257Z namespaces/openshift-cluster-version/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611326368Z namespaces/openshift-cluster-version/openshift-cluster-version.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61139182Z namespaces/openshift-cluster-version/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61143744Z namespaces/openshift-cluster-version/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611483552Z namespaces/openshift-cluster-version/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611522692Z namespaces/openshift-cluster-version/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611608724Z namespaces/openshift-cluster-version/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611689205Z namespaces/openshift-cluster-version/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611772407Z namespaces/openshift-cluster-version/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611825678Z namespaces/openshift-cluster-version/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611863569Z namespaces/openshift-cluster-version/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61192573Z namespaces/openshift-cluster-version/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.611965181Z namespaces/openshift-cluster-version/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612054183Z namespaces/openshift-cluster-version/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612111184Z namespaces/openshift-cluster-version/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612148065Z namespaces/openshift-cluster-version/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612266547Z namespaces/openshift-cluster-version/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612322478Z namespaces/openshift-cluster-version/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612330308Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612384539Z namespaces/openshift-cluster-version/coordination.k8s.io/leases/version.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612459451Z namespaces/openshift-cluster-version/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612510452Z namespaces/openshift-cluster-version/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612604123Z namespaces/openshift-cluster-version/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612689385Z namespaces/openshift-cluster-version/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612801367Z namespaces/openshift-cluster-version/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612881359Z namespaces/openshift-cluster-version/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.612960801Z namespaces/openshift-cluster-version/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613093993Z namespaces/openshift-cluster-version/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613301717Z namespaces/openshift-cluster-version/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61345673Z namespaces/openshift-cluster-version/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613501591Z namespaces/openshift-cluster-version/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613561602Z namespaces/openshift-cluster-version/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613610603Z namespaces/openshift-cluster-version/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613662474Z namespaces/openshift-cluster-version/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613703275Z namespaces/openshift-cluster-version/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613788697Z namespaces/openshift-cluster-version/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613872628Z namespaces/openshift-cluster-version/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.613916819Z namespaces/openshift-cluster-version/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61395176Z namespaces/openshift-cluster-version/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614022711Z namespaces/openshift-cluster-version/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614069632Z namespaces/openshift-cluster-version/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614122333Z namespaces/openshift-cluster-version/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614135553Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614281886Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614446489Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614628683Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.614766086Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61496104Z namespaces/openshift-cluster-version/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615291286Z namespaces/openshift-cluster-version/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615325156Z namespaces/openshift-cluster-version/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615400068Z namespaces/openshift-cluster-version/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615443459Z namespaces/openshift-cluster-version/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615547921Z namespaces/openshift-config-managed/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615597442Z namespaces/openshift-config-managed/openshift-config-managed.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615670743Z namespaces/openshift-config-managed/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615710554Z namespaces/openshift-config-managed/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615783145Z namespaces/openshift-config-managed/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615830196Z namespaces/openshift-config-managed/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.615911468Z namespaces/openshift-config-managed/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.61598998Z namespaces/openshift-config-managed/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616069651Z namespaces/openshift-config-managed/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616114252Z namespaces/openshift-config-managed/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616139342Z namespaces/openshift-config-managed/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616220554Z namespaces/openshift-config-managed/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616260595Z namespaces/openshift-config-managed/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616349056Z namespaces/openshift-config-managed/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616413588Z namespaces/openshift-config-managed/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616456989Z namespaces/openshift-config-managed/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6165454Z namespaces/openshift-config-managed/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616615402Z namespaces/openshift-config-managed/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.616663633Z namespaces/openshift-config-managed/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620020108Z namespaces/openshift-config-managed/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62009949Z namespaces/openshift-config-managed/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620208022Z namespaces/openshift-config-managed/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620294974Z namespaces/openshift-config-managed/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620375545Z namespaces/openshift-config-managed/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620546718Z namespaces/openshift-config-managed/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62064892Z namespaces/openshift-config-managed/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620712592Z namespaces/openshift-config-managed/core/configmaps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620769563Z namespaces/openshift-config-managed/core/configmaps/console-public.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620847255Z namespaces/openshift-config-managed/core/configmaps/openshift-network-features.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620903805Z namespaces/openshift-config-managed/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620930446Z namespaces/openshift-config-managed/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.620996287Z namespaces/openshift-config-managed/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621048238Z namespaces/openshift-config-managed/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621100369Z namespaces/openshift-config-managed/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62115075Z namespaces/openshift-config-managed/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621260192Z namespaces/openshift-config-managed/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621338444Z namespaces/openshift-config-managed/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621391455Z namespaces/openshift-config-managed/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621431976Z namespaces/openshift-config-managed/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621489697Z namespaces/openshift-config-managed/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621528148Z namespaces/openshift-config-managed/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621586669Z namespaces/openshift-config-managed/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621601929Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62164854Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621817653Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.621994987Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.622399115Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.622628749Z namespaces/openshift-config-managed/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.622906585Z namespaces/openshift-config-managed/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.622951625Z namespaces/openshift-config-managed/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623004766Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623013747Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623050487Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/rolebindings/openshift-network-public-role-binding.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623122069Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62318799Z namespaces/openshift-config-managed/rbac.authorization.k8s.io/roles/openshift-network-public-role.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623254411Z namespaces/openshift-config-managed/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623306423Z namespaces/openshift-config-managed/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623349883Z namespaces/openshift-config-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623358053Z namespaces/openshift-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623368334Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623420485Z namespaces/openshift-config-operator/monitoring.coreos.com/servicemonitors/config-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623493966Z namespaces/openshift-config-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623504556Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623566508Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62372288Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.623912294Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.624060577Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.624274981Z namespaces/openshift-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.624668329Z namespaces/openshift-config/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62472083Z namespaces/openshift-config/openshift-config.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.624780201Z namespaces/openshift-config/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.624798392Z namespaces/openshift-config/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.624884373Z namespaces/openshift-config/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.624915004Z namespaces/openshift-config/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625018806Z namespaces/openshift-config/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625093978Z namespaces/openshift-config/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625191249Z namespaces/openshift-config/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62524899Z namespaces/openshift-config/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625304931Z namespaces/openshift-config/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625365733Z namespaces/openshift-config/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625408764Z namespaces/openshift-config/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625498385Z namespaces/openshift-config/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625538046Z namespaces/openshift-config/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625587857Z namespaces/openshift-config/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625665809Z namespaces/openshift-config/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625700269Z namespaces/openshift-config/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62575228Z namespaces/openshift-config/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625897353Z namespaces/openshift-config/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.625976205Z namespaces/openshift-config/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626051436Z namespaces/openshift-config/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626134818Z namespaces/openshift-config/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62623809Z namespaces/openshift-config/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626404813Z namespaces/openshift-config/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626551726Z namespaces/openshift-config/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626581107Z namespaces/openshift-config/core/secrets/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626638028Z namespaces/openshift-config/core/secrets/pull-secret.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626673148Z namespaces/openshift-config/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62673421Z namespaces/openshift-config/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626787491Z namespaces/openshift-config/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626830871Z namespaces/openshift-config/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626892633Z namespaces/openshift-config/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.626925923Z namespaces/openshift-config/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627018485Z namespaces/openshift-config/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627098977Z namespaces/openshift-config/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627146787Z namespaces/openshift-config/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627209829Z namespaces/openshift-config/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62725759Z namespaces/openshift-config/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627313551Z namespaces/openshift-config/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627370052Z namespaces/openshift-config/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627383602Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627442903Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627595846Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6277775Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.627921402Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628240299Z namespaces/openshift-config/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628431272Z namespaces/openshift-config/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628473973Z namespaces/openshift-config/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628519654Z namespaces/openshift-config/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628567455Z namespaces/openshift-config/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628658877Z namespaces/openshift-console-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628707888Z namespaces/openshift-console-operator/openshift-console-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628779419Z namespaces/openshift-console-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6288228Z namespaces/openshift-console-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628894892Z namespaces/openshift-console-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.628937362Z namespaces/openshift-console-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629029994Z namespaces/openshift-console-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629161217Z namespaces/openshift-console-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62930373Z namespaces/openshift-console-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62934065Z namespaces/openshift-console-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629390931Z namespaces/openshift-console-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629436652Z namespaces/openshift-console-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629465663Z namespaces/openshift-console-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629560335Z namespaces/openshift-console-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629605566Z namespaces/openshift-console-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629642076Z namespaces/openshift-console-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629729678Z namespaces/openshift-console-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629826Z namespaces/openshift-console-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.62983429Z namespaces/openshift-console-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629877991Z namespaces/openshift-console-operator/coordination.k8s.io/leases/console-operator-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629953992Z namespaces/openshift-console-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.629987783Z namespaces/openshift-console-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.630679656Z namespaces/openshift-console-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.630762298Z namespaces/openshift-console-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631030553Z namespaces/openshift-console-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631111665Z namespaces/openshift-console-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631258098Z namespaces/openshift-console-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631398401Z namespaces/openshift-console-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631608665Z namespaces/openshift-console-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631649755Z namespaces/openshift-console-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631701787Z namespaces/openshift-console-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631742357Z namespaces/openshift-console-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631794278Z namespaces/openshift-console-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63185247Z namespaces/openshift-console-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63190702Z namespaces/openshift-console-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.631984082Z namespaces/openshift-console-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632067074Z namespaces/openshift-console-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632138245Z namespaces/openshift-console-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632212986Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632301138Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632343299Z namespaces/openshift-console-operator/monitoring.coreos.com/prometheusrules/cluster-monitoring-prometheus-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63240944Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632442301Z namespaces/openshift-console-operator/monitoring.coreos.com/servicemonitors/console-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632500852Z namespaces/openshift-console-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632529073Z namespaces/openshift-console-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632587304Z namespaces/openshift-console-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632597604Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632654845Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.632809258Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633093554Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633254807Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633457441Z namespaces/openshift-console-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633741906Z namespaces/openshift-console-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633751916Z namespaces/openshift-console-operator/pods/console-operator-565b7b569d-djbdl/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633788697Z namespaces/openshift-console-operator/pods/console-operator-565b7b569d-djbdl/console-operator-565b7b569d-djbdl.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633891169Z namespaces/openshift-console-operator/pods/console-operator-565b7b569d-djbdl/console-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63390109Z namespaces/openshift-console-operator/pods/console-operator-565b7b569d-djbdl/console-operator/console-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.633906519Z namespaces/openshift-console-operator/pods/console-operator-565b7b569d-djbdl/console-operator/console-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63394711Z namespaces/openshift-console-operator/pods/console-operator-565b7b569d-djbdl/console-operator/console-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634539802Z namespaces/openshift-console-operator/pods/console-operator-565b7b569d-djbdl/console-operator/console-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634579123Z namespaces/openshift-console-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634631924Z namespaces/openshift-console-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634685925Z namespaces/openshift-console-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634739126Z namespaces/openshift-console-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634797067Z namespaces/openshift-console-user-settings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634804427Z namespaces/openshift-console-user-settings/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634817527Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.634871558Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.635024161Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.635228245Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.635368618Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.635576232Z namespaces/openshift-console-user-settings/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.635791426Z namespaces/openshift-console/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.635838707Z namespaces/openshift-console/openshift-console.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636036161Z namespaces/openshift-console/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636091082Z namespaces/openshift-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636258236Z namespaces/openshift-console/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636280956Z namespaces/openshift-console/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636384178Z namespaces/openshift-console/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636553481Z namespaces/openshift-console/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636841747Z namespaces/openshift-console/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636873428Z namespaces/openshift-console/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.636922269Z namespaces/openshift-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63698555Z namespaces/openshift-console/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.637034481Z namespaces/openshift-console/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.637117702Z namespaces/openshift-console/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.637177443Z namespaces/openshift-console/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.637244645Z namespaces/openshift-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.637321206Z namespaces/openshift-console/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.637380167Z namespaces/openshift-console/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.637432659Z namespaces/openshift-console/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63803316Z namespaces/openshift-console/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.638118352Z namespaces/openshift-console/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.638492829Z namespaces/openshift-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.638570971Z namespaces/openshift-console/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.638771585Z namespaces/openshift-console/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.638904807Z namespaces/openshift-console/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639090841Z namespaces/openshift-console/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639147382Z namespaces/openshift-console/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639225554Z namespaces/openshift-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639295055Z namespaces/openshift-console/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639339496Z namespaces/openshift-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639397167Z namespaces/openshift-console/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639447748Z namespaces/openshift-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.63954055Z namespaces/openshift-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639619231Z namespaces/openshift-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639676552Z namespaces/openshift-console/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639719913Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639879066Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639907757Z namespaces/openshift-console/monitoring.coreos.com/servicemonitors/console.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.639983008Z namespaces/openshift-console/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640029849Z namespaces/openshift-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640107751Z namespaces/openshift-console/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640124301Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640184582Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640330785Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640511108Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640659271Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.640856395Z namespaces/openshift-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641053499Z namespaces/openshift-console/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641063059Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641120211Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/console-5ff5665d54-7bdbv.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641240013Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/console/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641251923Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/console/console/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641257293Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/console/console/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641296474Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/console/console/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641393406Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/console/console/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641460517Z namespaces/openshift-console/pods/console-5ff5665d54-7bdbv/console/console/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641502588Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641542129Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/downloads-5bc67dcd46-8n9sc.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64163591Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/download-server/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641644711Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/download-server/download-server/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641648771Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/download-server/download-server/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.641683162Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/download-server/download-server/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642041878Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/download-server/download-server/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64211251Z namespaces/openshift-console/pods/downloads-5bc67dcd46-8n9sc/download-server/download-server/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64212931Z namespaces/openshift-console/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642224402Z namespaces/openshift-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642280373Z namespaces/openshift-console/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642321304Z namespaces/openshift-console/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642453006Z namespaces/openshift-controller-manager-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642504178Z namespaces/openshift-controller-manager-operator/openshift-controller-manager-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642568229Z namespaces/openshift-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642599139Z namespaces/openshift-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642656701Z namespaces/openshift-controller-manager-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642688741Z namespaces/openshift-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642777893Z namespaces/openshift-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642857854Z namespaces/openshift-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.642941936Z namespaces/openshift-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643111019Z namespaces/openshift-controller-manager-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64316128Z namespaces/openshift-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643242422Z namespaces/openshift-controller-manager-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643282743Z namespaces/openshift-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643371784Z namespaces/openshift-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643423116Z namespaces/openshift-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643466296Z namespaces/openshift-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643552578Z namespaces/openshift-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643629409Z namespaces/openshift-controller-manager-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64367068Z namespaces/openshift-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643793443Z namespaces/openshift-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643872344Z namespaces/openshift-controller-manager-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.643951396Z namespaces/openshift-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644034557Z namespaces/openshift-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644113069Z namespaces/openshift-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644270792Z namespaces/openshift-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644449936Z namespaces/openshift-controller-manager-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644503806Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644536897Z namespaces/openshift-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644604379Z namespaces/openshift-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644644869Z namespaces/openshift-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644715591Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644755542Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644840263Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644924955Z namespaces/openshift-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.644978706Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645009547Z namespaces/openshift-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645062957Z namespaces/openshift-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645105008Z namespaces/openshift-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645142739Z namespaces/openshift-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645149179Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64521442Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645369194Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645546957Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.645820112Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646013716Z namespaces/openshift-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64623964Z namespaces/openshift-controller-manager-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646259781Z namespaces/openshift-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646316842Z namespaces/openshift-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646342813Z namespaces/openshift-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646439024Z namespaces/openshift-controller-manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646497036Z namespaces/openshift-controller-manager/openshift-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646552957Z namespaces/openshift-controller-manager/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646600858Z namespaces/openshift-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646636298Z namespaces/openshift-controller-manager/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646688939Z namespaces/openshift-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646772431Z namespaces/openshift-controller-manager/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646858383Z namespaces/openshift-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.646927684Z namespaces/openshift-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647018016Z namespaces/openshift-controller-manager/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647061716Z namespaces/openshift-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647134828Z namespaces/openshift-controller-manager/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647158118Z namespaces/openshift-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647294001Z namespaces/openshift-controller-manager/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647315292Z namespaces/openshift-controller-manager/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647371882Z namespaces/openshift-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647476655Z namespaces/openshift-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647525496Z namespaces/openshift-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647552766Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647610377Z namespaces/openshift-controller-manager/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647695489Z namespaces/openshift-controller-manager/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64774025Z namespaces/openshift-controller-manager/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647835312Z namespaces/openshift-controller-manager/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.647930364Z namespaces/openshift-controller-manager/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648020865Z namespaces/openshift-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648102067Z namespaces/openshift-controller-manager/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648182709Z namespaces/openshift-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648426623Z namespaces/openshift-controller-manager/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648608437Z namespaces/openshift-controller-manager/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648842041Z namespaces/openshift-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648887252Z namespaces/openshift-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648950813Z namespaces/openshift-controller-manager/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.648969154Z namespaces/openshift-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649026725Z namespaces/openshift-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649067736Z namespaces/openshift-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649157917Z namespaces/openshift-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64926287Z namespaces/openshift-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.64929937Z namespaces/openshift-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649348781Z namespaces/openshift-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649394512Z namespaces/openshift-controller-manager/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649438273Z namespaces/openshift-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649470064Z namespaces/openshift-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649485994Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649525725Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649683228Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.649860541Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650006074Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650227468Z namespaces/openshift-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650428272Z namespaces/openshift-controller-manager/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650450233Z namespaces/openshift-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650521154Z namespaces/openshift-controller-manager/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650550265Z namespaces/openshift-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650645686Z namespaces/openshift-dns-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650694318Z namespaces/openshift-dns-operator/openshift-dns-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650755209Z namespaces/openshift-dns-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65079982Z namespaces/openshift-dns-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650851981Z namespaces/openshift-dns-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650906062Z namespaces/openshift-dns-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.650985333Z namespaces/openshift-dns-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651070585Z namespaces/openshift-dns-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651149656Z namespaces/openshift-dns-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651228658Z namespaces/openshift-dns-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651277459Z namespaces/openshift-dns-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65132202Z namespaces/openshift-dns-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65135459Z namespaces/openshift-dns-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651443332Z namespaces/openshift-dns-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651502273Z namespaces/openshift-dns-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651552564Z namespaces/openshift-dns-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651632636Z namespaces/openshift-dns-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651709417Z namespaces/openshift-dns-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651755568Z namespaces/openshift-dns-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65184573Z namespaces/openshift-dns-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.651934532Z namespaces/openshift-dns-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652048384Z namespaces/openshift-dns-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652125025Z namespaces/openshift-dns-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652237368Z namespaces/openshift-dns-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6523767Z namespaces/openshift-dns-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652556204Z namespaces/openshift-dns-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652632695Z namespaces/openshift-dns-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652673616Z namespaces/openshift-dns-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652727487Z namespaces/openshift-dns-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652767558Z namespaces/openshift-dns-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652823819Z namespaces/openshift-dns-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65285952Z namespaces/openshift-dns-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.652948711Z namespaces/openshift-dns-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653031533Z namespaces/openshift-dns-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653074644Z namespaces/openshift-dns-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653119865Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653190756Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653238267Z namespaces/openshift-dns-operator/monitoring.coreos.com/prometheusrules/dns.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653294338Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653341599Z namespaces/openshift-dns-operator/monitoring.coreos.com/servicemonitors/dns-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65338666Z namespaces/openshift-dns-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653434141Z namespaces/openshift-dns-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653465682Z namespaces/openshift-dns-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653471822Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653532453Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653789828Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.653973592Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654106594Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654332689Z namespaces/openshift-dns-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654549133Z namespaces/openshift-dns-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654594354Z namespaces/openshift-dns-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654662395Z namespaces/openshift-dns-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654700826Z namespaces/openshift-dns-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654785918Z namespaces/openshift-dns/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654831928Z namespaces/openshift-dns/openshift-dns.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.654888049Z namespaces/openshift-dns/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65493233Z namespaces/openshift-dns/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.655628754Z namespaces/openshift-dns/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.655675725Z namespaces/openshift-dns/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.655825398Z namespaces/openshift-dns/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.655906479Z namespaces/openshift-dns/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.655986241Z namespaces/openshift-dns/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656041112Z namespaces/openshift-dns/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656077193Z namespaces/openshift-dns/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656139904Z namespaces/openshift-dns/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656188395Z namespaces/openshift-dns/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656379229Z namespaces/openshift-dns/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65643755Z namespaces/openshift-dns/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656488751Z namespaces/openshift-dns/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656565592Z namespaces/openshift-dns/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656634243Z namespaces/openshift-dns/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656682104Z namespaces/openshift-dns/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656772816Z namespaces/openshift-dns/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.656867048Z namespaces/openshift-dns/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.657092772Z namespaces/openshift-dns/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.657175264Z namespaces/openshift-dns/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65746286Z namespaces/openshift-dns/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.657582122Z namespaces/openshift-dns/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.657752835Z namespaces/openshift-dns/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.657817587Z namespaces/openshift-dns/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.657862047Z namespaces/openshift-dns/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.657923039Z namespaces/openshift-dns/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65797576Z namespaces/openshift-dns/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658041051Z namespaces/openshift-dns/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658085142Z namespaces/openshift-dns/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658185784Z namespaces/openshift-dns/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658292946Z namespaces/openshift-dns/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658359867Z namespaces/openshift-dns/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658418008Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65847413Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65851431Z namespaces/openshift-dns/monitoring.coreos.com/servicemonitors/dns-default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658571351Z namespaces/openshift-dns/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658602312Z namespaces/openshift-dns/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658652993Z namespaces/openshift-dns/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658660113Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658700594Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.658851137Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.65903502Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659381777Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659591741Z namespaces/openshift-dns/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659785535Z namespaces/openshift-dns/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659798785Z namespaces/openshift-dns/pods/dns-default-2hnhp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659861237Z namespaces/openshift-dns/pods/dns-default-2hnhp/dns-default-2hnhp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659946358Z namespaces/openshift-dns/pods/dns-default-2hnhp/dns/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659954898Z namespaces/openshift-dns/pods/dns-default-2hnhp/dns/dns/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.659961419Z namespaces/openshift-dns/pods/dns-default-2hnhp/dns/dns/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6600194Z namespaces/openshift-dns/pods/dns-default-2hnhp/dns/dns/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660125812Z namespaces/openshift-dns/pods/dns-default-2hnhp/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660217073Z namespaces/openshift-dns/pods/dns-default-2hnhp/dns/dns/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660249754Z namespaces/openshift-dns/pods/dns-default-2hnhp/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660257404Z namespaces/openshift-dns/pods/dns-default-2hnhp/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660263625Z namespaces/openshift-dns/pods/dns-default-2hnhp/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660323906Z namespaces/openshift-dns/pods/dns-default-2hnhp/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660428288Z namespaces/openshift-dns/pods/dns-default-2hnhp/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660498779Z namespaces/openshift-dns/pods/dns-default-2hnhp/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66053676Z namespaces/openshift-dns/pods/dns-default-8bgp6/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660584931Z namespaces/openshift-dns/pods/dns-default-8bgp6/dns-default-8bgp6.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660678153Z namespaces/openshift-dns/pods/dns-default-8bgp6/dns/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660693853Z namespaces/openshift-dns/pods/dns-default-8bgp6/dns/dns/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660704993Z namespaces/openshift-dns/pods/dns-default-8bgp6/dns/dns/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660724654Z namespaces/openshift-dns/pods/dns-default-8bgp6/dns/dns/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660824665Z namespaces/openshift-dns/pods/dns-default-8bgp6/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660900547Z namespaces/openshift-dns/pods/dns-default-8bgp6/dns/dns/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660941278Z namespaces/openshift-dns/pods/dns-default-8bgp6/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660950368Z namespaces/openshift-dns/pods/dns-default-8bgp6/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.660962738Z namespaces/openshift-dns/pods/dns-default-8bgp6/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661009039Z namespaces/openshift-dns/pods/dns-default-8bgp6/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661103991Z namespaces/openshift-dns/pods/dns-default-8bgp6/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661182292Z namespaces/openshift-dns/pods/dns-default-8bgp6/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661251604Z namespaces/openshift-dns/pods/dns-default-s686d/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661295295Z namespaces/openshift-dns/pods/dns-default-s686d/dns-default-s686d.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661376666Z namespaces/openshift-dns/pods/dns-default-s686d/dns/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661385816Z namespaces/openshift-dns/pods/dns-default-s686d/dns/dns/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661394467Z namespaces/openshift-dns/pods/dns-default-s686d/dns/dns/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661441028Z namespaces/openshift-dns/pods/dns-default-s686d/dns/dns/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66154361Z namespaces/openshift-dns/pods/dns-default-s686d/dns/dns/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661610641Z namespaces/openshift-dns/pods/dns-default-s686d/dns/dns/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661651021Z namespaces/openshift-dns/pods/dns-default-s686d/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661658772Z namespaces/openshift-dns/pods/dns-default-s686d/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661665232Z namespaces/openshift-dns/pods/dns-default-s686d/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661719613Z namespaces/openshift-dns/pods/dns-default-s686d/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661823775Z namespaces/openshift-dns/pods/dns-default-s686d/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661891136Z namespaces/openshift-dns/pods/dns-default-s686d/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.661936307Z namespaces/openshift-dns/pods/node-resolver-2dxpj/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662011008Z namespaces/openshift-dns/pods/node-resolver-2dxpj/node-resolver-2dxpj.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66209904Z namespaces/openshift-dns/pods/node-resolver-2dxpj/dns-node-resolver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66210763Z namespaces/openshift-dns/pods/node-resolver-2dxpj/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66211489Z namespaces/openshift-dns/pods/node-resolver-2dxpj/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662162782Z namespaces/openshift-dns/pods/node-resolver-2dxpj/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662286124Z namespaces/openshift-dns/pods/node-resolver-2dxpj/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662362775Z namespaces/openshift-dns/pods/node-resolver-2dxpj/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662545159Z namespaces/openshift-dns/pods/node-resolver-2wmz4/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662551809Z namespaces/openshift-dns/pods/node-resolver-2wmz4/node-resolver-2wmz4.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662559429Z namespaces/openshift-dns/pods/node-resolver-2wmz4/dns-node-resolver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66257123Z namespaces/openshift-dns/pods/node-resolver-2wmz4/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66257543Z namespaces/openshift-dns/pods/node-resolver-2wmz4/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662626751Z namespaces/openshift-dns/pods/node-resolver-2wmz4/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662713062Z namespaces/openshift-dns/pods/node-resolver-2wmz4/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662785084Z namespaces/openshift-dns/pods/node-resolver-2wmz4/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662824895Z namespaces/openshift-dns/pods/node-resolver-jfd48/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662862545Z namespaces/openshift-dns/pods/node-resolver-jfd48/node-resolver-jfd48.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662927017Z namespaces/openshift-dns/pods/node-resolver-jfd48/dns-node-resolver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662937617Z namespaces/openshift-dns/pods/node-resolver-jfd48/dns-node-resolver/dns-node-resolver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662945447Z namespaces/openshift-dns/pods/node-resolver-jfd48/dns-node-resolver/dns-node-resolver/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.662979027Z namespaces/openshift-dns/pods/node-resolver-jfd48/dns-node-resolver/dns-node-resolver/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663070199Z namespaces/openshift-dns/pods/node-resolver-jfd48/dns-node-resolver/dns-node-resolver/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663139091Z namespaces/openshift-dns/pods/node-resolver-jfd48/dns-node-resolver/dns-node-resolver/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663184822Z namespaces/openshift-dns/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663233072Z namespaces/openshift-dns/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663280413Z namespaces/openshift-dns/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663327214Z namespaces/openshift-dns/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663414726Z namespaces/openshift-etcd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663463167Z namespaces/openshift-etcd/openshift-etcd.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663519698Z namespaces/openshift-etcd/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663558589Z namespaces/openshift-etcd/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66362224Z namespaces/openshift-etcd/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663666901Z namespaces/openshift-etcd/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663751353Z namespaces/openshift-etcd/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663830124Z namespaces/openshift-etcd/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663910276Z namespaces/openshift-etcd/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663956397Z namespaces/openshift-etcd/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.663996807Z namespaces/openshift-etcd/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664047228Z namespaces/openshift-etcd/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664089029Z namespaces/openshift-etcd/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664185021Z namespaces/openshift-etcd/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664247672Z namespaces/openshift-etcd/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664293803Z namespaces/openshift-etcd/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664379995Z namespaces/openshift-etcd/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664450796Z namespaces/openshift-etcd/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664500077Z namespaces/openshift-etcd/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664603529Z namespaces/openshift-etcd/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664681261Z namespaces/openshift-etcd/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664763312Z namespaces/openshift-etcd/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664843634Z namespaces/openshift-etcd/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.664925945Z namespaces/openshift-etcd/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665046518Z namespaces/openshift-etcd/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665311913Z namespaces/openshift-etcd/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665380834Z namespaces/openshift-etcd/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665431995Z namespaces/openshift-etcd/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665474666Z namespaces/openshift-etcd/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665521317Z namespaces/openshift-etcd/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665568218Z namespaces/openshift-etcd/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665607619Z namespaces/openshift-etcd/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665691571Z namespaces/openshift-etcd/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665787262Z namespaces/openshift-etcd/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665839773Z namespaces/openshift-etcd/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665879564Z namespaces/openshift-etcd/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665929585Z namespaces/openshift-etcd/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.665979466Z namespaces/openshift-etcd/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.666019347Z namespaces/openshift-etcd/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.666029827Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.666072398Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.666253682Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.666501516Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.666664159Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.666866903Z namespaces/openshift-etcd/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667083288Z namespaces/openshift-etcd/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667129288Z namespaces/openshift-etcd/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66721613Z namespaces/openshift-etcd/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667264951Z namespaces/openshift-etcd/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667363563Z namespaces/openshift-host-network/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667412874Z namespaces/openshift-host-network/openshift-host-network.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667476645Z namespaces/openshift-host-network/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667519396Z namespaces/openshift-host-network/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667587128Z namespaces/openshift-host-network/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667629598Z namespaces/openshift-host-network/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66773028Z namespaces/openshift-host-network/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667820382Z namespaces/openshift-host-network/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667907264Z namespaces/openshift-host-network/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.667960695Z namespaces/openshift-host-network/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668001516Z namespaces/openshift-host-network/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66823532Z namespaces/openshift-host-network/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668280441Z namespaces/openshift-host-network/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668367353Z namespaces/openshift-host-network/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668418844Z namespaces/openshift-host-network/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668466415Z namespaces/openshift-host-network/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668546596Z namespaces/openshift-host-network/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668621468Z namespaces/openshift-host-network/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668670259Z namespaces/openshift-host-network/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668770941Z namespaces/openshift-host-network/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668848572Z namespaces/openshift-host-network/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.668931374Z namespaces/openshift-host-network/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669008275Z namespaces/openshift-host-network/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669095717Z namespaces/openshift-host-network/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66925943Z namespaces/openshift-host-network/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669356922Z namespaces/openshift-host-network/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669404343Z namespaces/openshift-host-network/core/resourcequotas/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669450844Z namespaces/openshift-host-network/core/resourcequotas/host-network-namespace-quotas.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669518485Z namespaces/openshift-host-network/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669568526Z namespaces/openshift-host-network/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669622627Z namespaces/openshift-host-network/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669670888Z namespaces/openshift-host-network/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669730119Z namespaces/openshift-host-network/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.66977873Z namespaces/openshift-host-network/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669857582Z namespaces/openshift-host-network/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669937714Z namespaces/openshift-host-network/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.669983224Z namespaces/openshift-host-network/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670027585Z namespaces/openshift-host-network/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670074576Z namespaces/openshift-host-network/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670114917Z namespaces/openshift-host-network/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670161688Z namespaces/openshift-host-network/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670205549Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.67025034Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670426023Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670605536Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670742149Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.670943093Z namespaces/openshift-host-network/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671144117Z namespaces/openshift-host-network/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671206258Z namespaces/openshift-host-network/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671261419Z namespaces/openshift-host-network/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.67130384Z namespaces/openshift-host-network/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671375991Z namespaces/openshift-image-registry/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671430992Z namespaces/openshift-image-registry/openshift-image-registry.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671509764Z namespaces/openshift-image-registry/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671539685Z namespaces/openshift-image-registry/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671584226Z namespaces/openshift-image-registry/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671613916Z namespaces/openshift-image-registry/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671731958Z namespaces/openshift-image-registry/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.671870371Z namespaces/openshift-image-registry/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672065135Z namespaces/openshift-image-registry/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672109466Z namespaces/openshift-image-registry/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672155907Z namespaces/openshift-image-registry/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672219578Z namespaces/openshift-image-registry/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672265549Z namespaces/openshift-image-registry/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672366301Z namespaces/openshift-image-registry/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672426332Z namespaces/openshift-image-registry/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672474123Z namespaces/openshift-image-registry/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672552434Z namespaces/openshift-image-registry/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672599266Z namespaces/openshift-image-registry/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672606606Z namespaces/openshift-image-registry/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672644116Z namespaces/openshift-image-registry/coordination.k8s.io/leases/openshift-master-controllers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672724158Z namespaces/openshift-image-registry/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.672772929Z namespaces/openshift-image-registry/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.673932132Z namespaces/openshift-image-registry/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.674035863Z namespaces/openshift-image-registry/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.674313459Z namespaces/openshift-image-registry/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.674397921Z namespaces/openshift-image-registry/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.674621965Z namespaces/openshift-image-registry/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.674740237Z namespaces/openshift-image-registry/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.674924851Z namespaces/openshift-image-registry/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.674982162Z namespaces/openshift-image-registry/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675011433Z namespaces/openshift-image-registry/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675075884Z namespaces/openshift-image-registry/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675103004Z namespaces/openshift-image-registry/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675177846Z namespaces/openshift-image-registry/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675221747Z namespaces/openshift-image-registry/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675310548Z namespaces/openshift-image-registry/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.67539327Z namespaces/openshift-image-registry/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675451221Z namespaces/openshift-image-registry/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675500452Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675563813Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675613784Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-operator-alerts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675709036Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/image-registry-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675798988Z namespaces/openshift-image-registry/monitoring.coreos.com/prometheusrules/imagestreams-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675858339Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.67591257Z namespaces/openshift-image-registry/monitoring.coreos.com/servicemonitors/image-registry.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.675976551Z namespaces/openshift-image-registry/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676026762Z namespaces/openshift-image-registry/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676060973Z namespaces/openshift-image-registry/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676100854Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676146175Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676328108Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676516402Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676654125Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.676848118Z namespaces/openshift-image-registry/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.677028772Z namespaces/openshift-image-registry/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.677037192Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.677075003Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/image-registry-6c56464c7b-b9pvp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.677182335Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/registry/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.677199645Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/registry/registry/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.677215436Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/registry/registry/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.677274237Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/registry/registry/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679331677Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/registry/registry/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679396108Z namespaces/openshift-image-registry/pods/image-registry-6c56464c7b-b9pvp/registry/registry/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679412259Z namespaces/openshift-image-registry/pods/node-ca-66rz6/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.67947157Z namespaces/openshift-image-registry/pods/node-ca-66rz6/node-ca-66rz6.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679543401Z namespaces/openshift-image-registry/pods/node-ca-66rz6/node-ca/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679553621Z namespaces/openshift-image-registry/pods/node-ca-66rz6/node-ca/node-ca/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679558571Z namespaces/openshift-image-registry/pods/node-ca-66rz6/node-ca/node-ca/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679622623Z namespaces/openshift-image-registry/pods/node-ca-66rz6/node-ca/node-ca/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679768006Z namespaces/openshift-image-registry/pods/node-ca-66rz6/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679839117Z namespaces/openshift-image-registry/pods/node-ca-66rz6/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679859727Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.679911088Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/node-ca-g2gl7.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.67999831Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/node-ca/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68000918Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/node-ca/node-ca/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680021231Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/node-ca/node-ca/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680067641Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/node-ca/node-ca/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680269505Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680336957Z namespaces/openshift-image-registry/pods/node-ca-g2gl7/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680359887Z namespaces/openshift-image-registry/pods/node-ca-x48x8/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680423648Z namespaces/openshift-image-registry/pods/node-ca-x48x8/node-ca-x48x8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680483509Z namespaces/openshift-image-registry/pods/node-ca-x48x8/node-ca/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68049387Z namespaces/openshift-image-registry/pods/node-ca-x48x8/node-ca/node-ca/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68050728Z namespaces/openshift-image-registry/pods/node-ca-x48x8/node-ca/node-ca/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680554211Z namespaces/openshift-image-registry/pods/node-ca-x48x8/node-ca/node-ca/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680716994Z namespaces/openshift-image-registry/pods/node-ca-x48x8/node-ca/node-ca/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680782965Z namespaces/openshift-image-registry/pods/node-ca-x48x8/node-ca/node-ca/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680842787Z namespaces/openshift-image-registry/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680887287Z namespaces/openshift-image-registry/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680953978Z namespaces/openshift-image-registry/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.680979579Z namespaces/openshift-image-registry/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68103582Z namespaces/openshift-infra/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68104572Z namespaces/openshift-infra/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68105168Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.681085651Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.681250984Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.681429978Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.681572871Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.681768925Z namespaces/openshift-infra/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68202709Z namespaces/openshift-ingress-canary/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682083191Z namespaces/openshift-ingress-canary/openshift-ingress-canary.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682183403Z namespaces/openshift-ingress-canary/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682306095Z namespaces/openshift-ingress-canary/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682394697Z namespaces/openshift-ingress-canary/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682445708Z namespaces/openshift-ingress-canary/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68256269Z namespaces/openshift-ingress-canary/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682638462Z namespaces/openshift-ingress-canary/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682722373Z namespaces/openshift-ingress-canary/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682852276Z namespaces/openshift-ingress-canary/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682890407Z namespaces/openshift-ingress-canary/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682961938Z namespaces/openshift-ingress-canary/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.682997489Z namespaces/openshift-ingress-canary/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68308355Z namespaces/openshift-ingress-canary/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683119631Z namespaces/openshift-ingress-canary/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683190492Z namespaces/openshift-ingress-canary/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683286524Z namespaces/openshift-ingress-canary/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683370106Z namespaces/openshift-ingress-canary/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683412647Z namespaces/openshift-ingress-canary/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683517829Z namespaces/openshift-ingress-canary/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68360845Z namespaces/openshift-ingress-canary/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683757123Z namespaces/openshift-ingress-canary/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683830385Z namespaces/openshift-ingress-canary/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.683996798Z namespaces/openshift-ingress-canary/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68411709Z namespaces/openshift-ingress-canary/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684308294Z namespaces/openshift-ingress-canary/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684352625Z namespaces/openshift-ingress-canary/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684400796Z namespaces/openshift-ingress-canary/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684457777Z namespaces/openshift-ingress-canary/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684493008Z namespaces/openshift-ingress-canary/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684566389Z namespaces/openshift-ingress-canary/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68460618Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684696852Z namespaces/openshift-ingress-canary/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684775423Z namespaces/openshift-ingress-canary/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684826574Z namespaces/openshift-ingress-canary/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684861755Z namespaces/openshift-ingress-canary/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684912876Z namespaces/openshift-ingress-canary/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.684941247Z namespaces/openshift-ingress-canary/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.685001418Z namespaces/openshift-ingress-canary/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.685012838Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.685068509Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.685253173Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.685428206Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.685577949Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.685764783Z namespaces/openshift-ingress-canary/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686027568Z namespaces/openshift-ingress-canary/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686037408Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686078199Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/ingress-canary-kglbm.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6861631Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/serve-healthcheck-canary/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686197361Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686202251Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686248092Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686350274Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686430896Z namespaces/openshift-ingress-canary/pods/ingress-canary-kglbm/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686490567Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686512707Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/ingress-canary-nn2gp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686600149Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/serve-healthcheck-canary/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686608659Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686619819Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.6866678Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686761262Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.686829673Z namespaces/openshift-ingress-canary/pods/ingress-canary-nn2gp/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68714107Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68716013Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/ingress-canary-w6n4p.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687290282Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/serve-healthcheck-canary/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687311613Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/serve-healthcheck-canary/serve-healthcheck-canary/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687324603Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/serve-healthcheck-canary/serve-healthcheck-canary/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687354624Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/serve-healthcheck-canary/serve-healthcheck-canary/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687447725Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687524387Z namespaces/openshift-ingress-canary/pods/ingress-canary-w6n4p/serve-healthcheck-canary/serve-healthcheck-canary/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687547457Z namespaces/openshift-ingress-canary/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687595618Z namespaces/openshift-ingress-canary/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687648569Z namespaces/openshift-ingress-canary/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.68769423Z namespaces/openshift-ingress-canary/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687800222Z namespaces/openshift-ingress-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687849013Z namespaces/openshift-ingress-operator/openshift-ingress-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687911385Z namespaces/openshift-ingress-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.687947145Z namespaces/openshift-ingress-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688019756Z namespaces/openshift-ingress-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688054397Z namespaces/openshift-ingress-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688302612Z namespaces/openshift-ingress-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688379054Z namespaces/openshift-ingress-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688462335Z namespaces/openshift-ingress-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688519126Z namespaces/openshift-ingress-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688555777Z namespaces/openshift-ingress-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688623228Z namespaces/openshift-ingress-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688666519Z namespaces/openshift-ingress-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688744121Z namespaces/openshift-ingress-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688791922Z namespaces/openshift-ingress-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688838153Z namespaces/openshift-ingress-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688915474Z namespaces/openshift-ingress-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.688982835Z namespaces/openshift-ingress-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.689024346Z namespaces/openshift-ingress-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.689922194Z namespaces/openshift-ingress-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690006075Z namespaces/openshift-ingress-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690137468Z namespaces/openshift-ingress-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69025448Z namespaces/openshift-ingress-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690339692Z namespaces/openshift-ingress-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690496185Z namespaces/openshift-ingress-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690679739Z namespaces/openshift-ingress-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69073516Z namespaces/openshift-ingress-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690785211Z namespaces/openshift-ingress-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690844612Z namespaces/openshift-ingress-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690891763Z namespaces/openshift-ingress-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690948404Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.690958724Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691005405Z namespaces/openshift-ingress-operator/ingress.operator.openshift.io/dnsrecords/default-wildcard.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691079766Z namespaces/openshift-ingress-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691117197Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691297171Z namespaces/openshift-ingress-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691378212Z namespaces/openshift-ingress-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691439894Z namespaces/openshift-ingress-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691490124Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691544405Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691593477Z namespaces/openshift-ingress-operator/monitoring.coreos.com/prometheusrules/ingress-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691672698Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691721669Z namespaces/openshift-ingress-operator/monitoring.coreos.com/servicemonitors/ingress-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69178226Z namespaces/openshift-ingress-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691824001Z namespaces/openshift-ingress-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691885572Z namespaces/openshift-ingress-operator/operator.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691892632Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.691934923Z namespaces/openshift-ingress-operator/operator.openshift.io/ingresscontrollers/default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.692014565Z namespaces/openshift-ingress-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.692025235Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.692074986Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.692238109Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.692429273Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.692627367Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.692824181Z namespaces/openshift-ingress-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693045885Z namespaces/openshift-ingress-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693078735Z namespaces/openshift-ingress-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693262099Z namespaces/openshift-ingress-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693283159Z namespaces/openshift-ingress-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693399572Z namespaces/openshift-ingress/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693450093Z namespaces/openshift-ingress/openshift-ingress.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693520794Z namespaces/openshift-ingress/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693563225Z namespaces/openshift-ingress/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693624586Z namespaces/openshift-ingress/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693652187Z namespaces/openshift-ingress/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693743908Z namespaces/openshift-ingress/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.693966913Z namespaces/openshift-ingress/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694182277Z namespaces/openshift-ingress/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694242368Z namespaces/openshift-ingress/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694286809Z namespaces/openshift-ingress/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69435286Z namespaces/openshift-ingress/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694410381Z namespaces/openshift-ingress/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694503443Z namespaces/openshift-ingress/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694560864Z namespaces/openshift-ingress/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694607165Z namespaces/openshift-ingress/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694686237Z namespaces/openshift-ingress/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694745218Z namespaces/openshift-ingress/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694763339Z namespaces/openshift-ingress/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694802749Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-ca-openshift-gateway.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694894011Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-deployment-openshift-gateway.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.694973663Z namespaces/openshift-ingress/coordination.k8s.io/leases/istio-gateway-status-leader-openshift-gateway.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.695072025Z namespaces/openshift-ingress/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.695101005Z namespaces/openshift-ingress/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.695565714Z namespaces/openshift-ingress/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.695672946Z namespaces/openshift-ingress/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.695912751Z namespaces/openshift-ingress/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696002923Z namespaces/openshift-ingress/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696279948Z namespaces/openshift-ingress/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69640264Z namespaces/openshift-ingress/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696598114Z namespaces/openshift-ingress/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696699826Z namespaces/openshift-ingress/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696726587Z namespaces/openshift-ingress/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696823529Z namespaces/openshift-ingress/gateway.networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696830999Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69688072Z namespaces/openshift-ingress/gateway.networking.k8s.io/gateways/openshift-ai-inference.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696941431Z namespaces/openshift-ingress/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.696971271Z namespaces/openshift-ingress/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697049403Z namespaces/openshift-ingress/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697099724Z namespaces/openshift-ingress/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697182916Z namespaces/openshift-ingress/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697279668Z namespaces/openshift-ingress/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697328309Z namespaces/openshift-ingress/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69738432Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697439201Z namespaces/openshift-ingress/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697488372Z namespaces/openshift-ingress/monitoring.coreos.com/podmonitors/openshift-ai-inference-metrics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697552813Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697601984Z namespaces/openshift-ingress/monitoring.coreos.com/servicemonitors/router-default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697649755Z namespaces/openshift-ingress/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697700446Z namespaces/openshift-ingress/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697736877Z namespaces/openshift-ingress/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697756537Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697801898Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.697953691Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.698268647Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.69842287Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.698620664Z namespaces/openshift-ingress/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.698824328Z namespaces/openshift-ingress/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.698835468Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.698866039Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/istiod-openshift-gateway-94bb8fbfd-zltzb.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.699008591Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.699019722Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.699027562Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.699066912Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70458664Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.704648041Z namespaces/openshift-ingress/pods/istiod-openshift-gateway-94bb8fbfd-zltzb/discovery/discovery/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.704707523Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.704739803Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70510293Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70511165Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70511665Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.705158071Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.705779063Z namespaces/openshift-ingress/pods/openshift-ai-inference-openshift-default-655f7c58b4-jfpwp/istio-proxy/istio-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.705861185Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.705899576Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router-default-5f6bfd579b-gk6x5.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706038419Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706045539Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706051709Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70610303Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706347665Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706425956Z namespaces/openshift-ingress/pods/router-default-5f6bfd579b-gk6x5/router/router/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706477977Z namespaces/openshift-ingress/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706513558Z namespaces/openshift-ingress/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706579689Z namespaces/openshift-ingress/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70660983Z namespaces/openshift-ingress/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706718992Z namespaces/openshift-insights/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706758163Z namespaces/openshift-insights/openshift-insights.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706838984Z namespaces/openshift-insights/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706878125Z namespaces/openshift-insights/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706947306Z namespaces/openshift-insights/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.706979577Z namespaces/openshift-insights/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70711079Z namespaces/openshift-insights/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707237472Z namespaces/openshift-insights/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707338664Z namespaces/openshift-insights/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707397675Z namespaces/openshift-insights/apps/deployments/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707438596Z namespaces/openshift-insights/apps/deployments/insights-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707509968Z namespaces/openshift-insights/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707531658Z namespaces/openshift-insights/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707597939Z namespaces/openshift-insights/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.70764253Z namespaces/openshift-insights/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707723701Z namespaces/openshift-insights/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707781443Z namespaces/openshift-insights/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707832394Z namespaces/openshift-insights/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707907735Z namespaces/openshift-insights/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.707983687Z namespaces/openshift-insights/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.708033478Z namespaces/openshift-insights/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.708962906Z namespaces/openshift-insights/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.709041877Z namespaces/openshift-insights/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.709362253Z namespaces/openshift-insights/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.709489166Z namespaces/openshift-insights/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.709774752Z namespaces/openshift-insights/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.709873354Z namespaces/openshift-insights/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710013246Z namespaces/openshift-insights/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710079547Z namespaces/openshift-insights/core/configmaps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710128668Z namespaces/openshift-insights/core/configmaps/service-ca-bundle.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71021368Z namespaces/openshift-insights/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710252051Z namespaces/openshift-insights/core/serviceaccounts/gather.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710345763Z namespaces/openshift-insights/core/serviceaccounts/operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710420244Z namespaces/openshift-insights/core/services/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710454835Z namespaces/openshift-insights/core/services/metrics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710530466Z namespaces/openshift-insights/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710559717Z namespaces/openshift-insights/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710634348Z namespaces/openshift-insights/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710682419Z namespaces/openshift-insights/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71074132Z namespaces/openshift-insights/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710780361Z namespaces/openshift-insights/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710865143Z namespaces/openshift-insights/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710943364Z namespaces/openshift-insights/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.710998856Z namespaces/openshift-insights/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711046657Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711112138Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711157229Z namespaces/openshift-insights/monitoring.coreos.com/prometheusrules/insights-prometheus-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7112458Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711294711Z namespaces/openshift-insights/monitoring.coreos.com/servicemonitors/insights-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711357932Z namespaces/openshift-insights/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711389853Z namespaces/openshift-insights/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711447044Z namespaces/openshift-insights/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711478675Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711524416Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71171969Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.711944324Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712089157Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712313241Z namespaces/openshift-insights/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712554466Z namespaces/openshift-insights/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712563146Z namespaces/openshift-insights/pods/insights-operator-555d9d7885-rm45p/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712592377Z namespaces/openshift-insights/pods/insights-operator-555d9d7885-rm45p/insights-operator-555d9d7885-rm45p.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712709969Z namespaces/openshift-insights/pods/insights-operator-555d9d7885-rm45p/insights-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712720269Z namespaces/openshift-insights/pods/insights-operator-555d9d7885-rm45p/insights-operator/insights-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.712732399Z namespaces/openshift-insights/pods/insights-operator-555d9d7885-rm45p/insights-operator/insights-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71276922Z namespaces/openshift-insights/pods/insights-operator-555d9d7885-rm45p/insights-operator/insights-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713178498Z namespaces/openshift-insights/pods/insights-operator-555d9d7885-rm45p/insights-operator/insights-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713575276Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713596926Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/insights-runtime-extractor-fqs2h.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713723029Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713733499Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/exporter/exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713741629Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/exporter/exporter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713752759Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/exporter/exporter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713857492Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713928363Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/exporter/exporter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713965554Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/extractor/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713973364Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/extractor/extractor/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.713978334Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/extractor/extractor/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714016275Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/extractor/extractor/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714125327Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714195008Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/extractor/extractor/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71429878Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714308Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714320621Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714395392Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714481004Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714550485Z namespaces/openshift-insights/pods/insights-runtime-extractor-fqs2h/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714597346Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714635576Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/insights-runtime-extractor-h8tvt.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714730038Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714743829Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/exporter/exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714758479Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/exporter/exporter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71479108Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/exporter/exporter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714885261Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714952173Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/exporter/exporter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714968053Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/extractor/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714991383Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/extractor/extractor/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.714997904Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/extractor/extractor/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715045604Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/extractor/extractor/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715141006Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715230938Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/extractor/extractor/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715251679Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715471573Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715479873Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715521074Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715608726Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715681627Z namespaces/openshift-insights/pods/insights-runtime-extractor-h8tvt/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715709208Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715758838Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/insights-runtime-extractor-j7t54.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71584337Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7158546Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/exporter/exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71585898Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/exporter/exporter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715893911Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/exporter/exporter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.715984773Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/exporter/exporter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716053564Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/exporter/exporter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716087635Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/extractor/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716103885Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/extractor/extractor/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716110636Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/extractor/extractor/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716146806Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/extractor/extractor/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716272888Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/extractor/extractor/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71633847Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/extractor/extractor/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71635987Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716368881Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71637477Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716419731Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716517993Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716582045Z namespaces/openshift-insights/pods/insights-runtime-extractor-j7t54/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716630586Z namespaces/openshift-insights/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716643566Z namespaces/openshift-insights/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716705927Z namespaces/openshift-insights/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716750678Z namespaces/openshift-insights/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716803779Z namespaces/openshift-keda/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716811879Z namespaces/openshift-keda/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716815989Z namespaces/openshift-keda/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71685813Z namespaces/openshift-keda/coordination.k8s.io/leases/olm-operator.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716946402Z namespaces/openshift-keda/coordination.k8s.io/leases/operator.keda.sh.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.716999833Z namespaces/openshift-keda/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717008693Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717041423Z namespaces/openshift-keda/monitoring.coreos.com/podmonitors/keda-olm-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717103125Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717135055Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-admission-webhooks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717257928Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-metrics-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71734248Z namespaces/openshift-keda/monitoring.coreos.com/servicemonitors/keda-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717394021Z namespaces/openshift-keda/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71740167Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717457002Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717650215Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.717847059Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/custom-metrics-autoscaler.v2.19.0-2.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.718049873Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.718306568Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.718474732Z namespaces/openshift-keda/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.718685106Z namespaces/openshift-keda/operators.coreos.com/installplans/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.718743137Z namespaces/openshift-keda/operators.coreos.com/installplans/install-7xfw9.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.71888386Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.718941131Z namespaces/openshift-keda/operators.coreos.com/operatorconditions/custom-metrics-autoscaler.v2.19.0-2.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719008352Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719050163Z namespaces/openshift-keda/operators.coreos.com/operatorgroups/openshift-keda.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719129555Z namespaces/openshift-keda/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719188085Z namespaces/openshift-keda/operators.coreos.com/subscriptions/openshift-custom-metrics-autoscaler-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719331028Z namespaces/openshift-kube-apiserver-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719391149Z namespaces/openshift-kube-apiserver-operator/openshift-kube-apiserver-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719467441Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719499592Z namespaces/openshift-kube-apiserver-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719574233Z namespaces/openshift-kube-apiserver-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719610294Z namespaces/openshift-kube-apiserver-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719691855Z namespaces/openshift-kube-apiserver-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719776637Z namespaces/openshift-kube-apiserver-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719857038Z namespaces/openshift-kube-apiserver-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7199187Z namespaces/openshift-kube-apiserver-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.719953971Z namespaces/openshift-kube-apiserver-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720020122Z namespaces/openshift-kube-apiserver-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720065843Z namespaces/openshift-kube-apiserver-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720176325Z namespaces/openshift-kube-apiserver-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720274987Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720337248Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72043588Z namespaces/openshift-kube-apiserver-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720533592Z namespaces/openshift-kube-apiserver-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720592143Z namespaces/openshift-kube-apiserver-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720728516Z namespaces/openshift-kube-apiserver-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.720831258Z namespaces/openshift-kube-apiserver-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7209349Z namespaces/openshift-kube-apiserver-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721035592Z namespaces/openshift-kube-apiserver-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721141174Z namespaces/openshift-kube-apiserver-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721314857Z namespaces/openshift-kube-apiserver-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721504851Z namespaces/openshift-kube-apiserver-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721567882Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721621783Z namespaces/openshift-kube-apiserver-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721670794Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721708985Z namespaces/openshift-kube-apiserver-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721783006Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.721830857Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72247783Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72250597Z namespaces/openshift-kube-apiserver-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72251301Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72251839Z namespaces/openshift-kube-apiserver-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.722530301Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.722535531Z namespaces/openshift-kube-apiserver-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.722547751Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.722552751Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.722562331Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.723149313Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.723740394Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.723751965Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.723764295Z namespaces/openshift-kube-apiserver-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.723938318Z namespaces/openshift-kube-apiserver-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.723984329Z namespaces/openshift-kube-apiserver-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72403004Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72405074Z namespaces/openshift-kube-apiserver-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724186353Z namespaces/openshift-kube-apiserver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724253484Z namespaces/openshift-kube-apiserver/openshift-kube-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724326116Z namespaces/openshift-kube-apiserver/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724375857Z namespaces/openshift-kube-apiserver/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724434218Z namespaces/openshift-kube-apiserver/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724480779Z namespaces/openshift-kube-apiserver/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7245592Z namespaces/openshift-kube-apiserver/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724638602Z namespaces/openshift-kube-apiserver/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724720024Z namespaces/openshift-kube-apiserver/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724775625Z namespaces/openshift-kube-apiserver/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724822766Z namespaces/openshift-kube-apiserver/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724883407Z namespaces/openshift-kube-apiserver/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.724929168Z namespaces/openshift-kube-apiserver/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725011019Z namespaces/openshift-kube-apiserver/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72505771Z namespaces/openshift-kube-apiserver/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725103751Z namespaces/openshift-kube-apiserver/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725190693Z namespaces/openshift-kube-apiserver/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725272794Z namespaces/openshift-kube-apiserver/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725314845Z namespaces/openshift-kube-apiserver/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725419947Z namespaces/openshift-kube-apiserver/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725494479Z namespaces/openshift-kube-apiserver/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72557537Z namespaces/openshift-kube-apiserver/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725652872Z namespaces/openshift-kube-apiserver/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725730053Z namespaces/openshift-kube-apiserver/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.725853066Z namespaces/openshift-kube-apiserver/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726002729Z namespaces/openshift-kube-apiserver/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72606939Z namespaces/openshift-kube-apiserver/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726110631Z namespaces/openshift-kube-apiserver/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726159532Z namespaces/openshift-kube-apiserver/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726237803Z namespaces/openshift-kube-apiserver/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726297484Z namespaces/openshift-kube-apiserver/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726346375Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726438397Z namespaces/openshift-kube-apiserver/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726524569Z namespaces/openshift-kube-apiserver/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7265814Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726628961Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726688672Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726734753Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/api-usage.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726848165Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/prometheusrules/podsecurity.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726931017Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.726977358Z namespaces/openshift-kube-apiserver/monitoring.coreos.com/servicemonitors/openshift-kube-apiserver.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727049889Z namespaces/openshift-kube-apiserver/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.72706811Z namespaces/openshift-kube-apiserver/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727129761Z namespaces/openshift-kube-apiserver/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727140801Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727201182Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727357195Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727544329Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727682802Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.727880735Z namespaces/openshift-kube-apiserver/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728079019Z namespaces/openshift-kube-apiserver/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728094459Z namespaces/openshift-kube-apiserver/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728157641Z namespaces/openshift-kube-apiserver/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728288853Z namespaces/openshift-kube-apiserver/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728373265Z namespaces/openshift-kube-controller-manager-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728430636Z namespaces/openshift-kube-controller-manager-operator/openshift-kube-controller-manager-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728499358Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728537598Z namespaces/openshift-kube-controller-manager-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728609269Z namespaces/openshift-kube-controller-manager-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728654541Z namespaces/openshift-kube-controller-manager-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728735432Z namespaces/openshift-kube-controller-manager-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728816564Z namespaces/openshift-kube-controller-manager-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728896265Z namespaces/openshift-kube-controller-manager-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728928236Z namespaces/openshift-kube-controller-manager-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.728981027Z namespaces/openshift-kube-controller-manager-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729038018Z namespaces/openshift-kube-controller-manager-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729071638Z namespaces/openshift-kube-controller-manager-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729181991Z namespaces/openshift-kube-controller-manager-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729224401Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729284393Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729370744Z namespaces/openshift-kube-controller-manager-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729455646Z namespaces/openshift-kube-controller-manager-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729504697Z namespaces/openshift-kube-controller-manager-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729821113Z namespaces/openshift-kube-controller-manager-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729901925Z namespaces/openshift-kube-controller-manager-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.729982596Z namespaces/openshift-kube-controller-manager-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730061978Z namespaces/openshift-kube-controller-manager-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730140939Z namespaces/openshift-kube-controller-manager-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730297293Z namespaces/openshift-kube-controller-manager-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730478186Z namespaces/openshift-kube-controller-manager-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730557238Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730604019Z namespaces/openshift-kube-controller-manager-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73065994Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730700201Z namespaces/openshift-kube-controller-manager-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730754311Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730796152Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730879164Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.730955026Z namespaces/openshift-kube-controller-manager-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731003336Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731046027Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731099568Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731139559Z namespaces/openshift-kube-controller-manager-operator/monitoring.coreos.com/servicemonitors/kube-controller-manager-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731225651Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731272272Z namespaces/openshift-kube-controller-manager-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731324563Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731331983Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731374944Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731518477Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73170357Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.731844403Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732039007Z namespaces/openshift-kube-controller-manager-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732261841Z namespaces/openshift-kube-controller-manager-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732304742Z namespaces/openshift-kube-controller-manager-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732360803Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732399984Z namespaces/openshift-kube-controller-manager-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732460475Z namespaces/openshift-kube-controller-manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732503796Z namespaces/openshift-kube-controller-manager/openshift-kube-controller-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732562717Z namespaces/openshift-kube-controller-manager/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732605998Z namespaces/openshift-kube-controller-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732660039Z namespaces/openshift-kube-controller-manager/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73270715Z namespaces/openshift-kube-controller-manager/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732791831Z namespaces/openshift-kube-controller-manager/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732867333Z namespaces/openshift-kube-controller-manager/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.732943424Z namespaces/openshift-kube-controller-manager/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733003766Z namespaces/openshift-kube-controller-manager/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733027896Z namespaces/openshift-kube-controller-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733074307Z namespaces/openshift-kube-controller-manager/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733111488Z namespaces/openshift-kube-controller-manager/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73323106Z namespaces/openshift-kube-controller-manager/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733284201Z namespaces/openshift-kube-controller-manager/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733318992Z namespaces/openshift-kube-controller-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733404493Z namespaces/openshift-kube-controller-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733454374Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733468684Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733500825Z namespaces/openshift-kube-controller-manager/coordination.k8s.io/leases/cluster-policy-controller-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733561356Z namespaces/openshift-kube-controller-manager/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733603517Z namespaces/openshift-kube-controller-manager/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.733703829Z namespaces/openshift-kube-controller-manager/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73377663Z namespaces/openshift-kube-controller-manager/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73426726Z namespaces/openshift-kube-controller-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734344021Z namespaces/openshift-kube-controller-manager/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734435413Z namespaces/openshift-kube-controller-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734557576Z namespaces/openshift-kube-controller-manager/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734716629Z namespaces/openshift-kube-controller-manager/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73475933Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734800501Z namespaces/openshift-kube-controller-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734845871Z namespaces/openshift-kube-controller-manager/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734894312Z namespaces/openshift-kube-controller-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.734954684Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735000934Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735084586Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735163058Z namespaces/openshift-kube-controller-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735352751Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735371082Z namespaces/openshift-kube-controller-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735437673Z namespaces/openshift-kube-controller-manager/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735483334Z namespaces/openshift-kube-controller-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735521664Z namespaces/openshift-kube-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735530035Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735572045Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735715638Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.735894622Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736030525Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736254429Z namespaces/openshift-kube-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736443882Z namespaces/openshift-kube-controller-manager/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736491314Z namespaces/openshift-kube-controller-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736535844Z namespaces/openshift-kube-controller-manager/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736554735Z namespaces/openshift-kube-controller-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736632926Z namespaces/openshift-kube-scheduler-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736667257Z namespaces/openshift-kube-scheduler-operator/openshift-kube-scheduler-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736731508Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736761409Z namespaces/openshift-kube-scheduler-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73681782Z namespaces/openshift-kube-scheduler-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736853601Z namespaces/openshift-kube-scheduler-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.736937592Z namespaces/openshift-kube-scheduler-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737013644Z namespaces/openshift-kube-scheduler-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737092745Z namespaces/openshift-kube-scheduler-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737136376Z namespaces/openshift-kube-scheduler-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737205677Z namespaces/openshift-kube-scheduler-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737260478Z namespaces/openshift-kube-scheduler-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737282719Z namespaces/openshift-kube-scheduler-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737371201Z namespaces/openshift-kube-scheduler-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737417432Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737452742Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737531834Z namespaces/openshift-kube-scheduler-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737584125Z namespaces/openshift-kube-scheduler-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737620606Z namespaces/openshift-kube-scheduler-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737733258Z namespaces/openshift-kube-scheduler-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737808129Z namespaces/openshift-kube-scheduler-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737885831Z namespaces/openshift-kube-scheduler-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.737968563Z namespaces/openshift-kube-scheduler-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738042344Z namespaces/openshift-kube-scheduler-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738179856Z namespaces/openshift-kube-scheduler-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73835236Z namespaces/openshift-kube-scheduler-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738410251Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738454072Z namespaces/openshift-kube-scheduler-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738509303Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738539154Z namespaces/openshift-kube-scheduler-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738605165Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738654906Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738740238Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738817809Z namespaces/openshift-kube-scheduler-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73886768Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.73888687Z namespaces/openshift-kube-scheduler-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738957022Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.738997832Z namespaces/openshift-kube-scheduler-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739048053Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739057244Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739096164Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739282088Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739464702Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739609775Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739803328Z namespaces/openshift-kube-scheduler-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.739987962Z namespaces/openshift-kube-scheduler-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740020352Z namespaces/openshift-kube-scheduler-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740074904Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740119385Z namespaces/openshift-kube-scheduler-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740190066Z namespaces/openshift-kube-scheduler/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740235327Z namespaces/openshift-kube-scheduler/openshift-kube-scheduler.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740302818Z namespaces/openshift-kube-scheduler/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740335459Z namespaces/openshift-kube-scheduler/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74039054Z namespaces/openshift-kube-scheduler/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74040963Z namespaces/openshift-kube-scheduler/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740510812Z namespaces/openshift-kube-scheduler/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740585354Z namespaces/openshift-kube-scheduler/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740668375Z namespaces/openshift-kube-scheduler/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740707186Z namespaces/openshift-kube-scheduler/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740747317Z namespaces/openshift-kube-scheduler/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740799938Z namespaces/openshift-kube-scheduler/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740836808Z namespaces/openshift-kube-scheduler/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74091737Z namespaces/openshift-kube-scheduler/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740955801Z namespaces/openshift-kube-scheduler/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.740995431Z namespaces/openshift-kube-scheduler/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741076403Z namespaces/openshift-kube-scheduler/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741130644Z namespaces/openshift-kube-scheduler/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741181285Z namespaces/openshift-kube-scheduler/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741306268Z namespaces/openshift-kube-scheduler/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741371769Z namespaces/openshift-kube-scheduler/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741457631Z namespaces/openshift-kube-scheduler/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741535932Z namespaces/openshift-kube-scheduler/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741624094Z namespaces/openshift-kube-scheduler/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741738976Z namespaces/openshift-kube-scheduler/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741898109Z namespaces/openshift-kube-scheduler/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74194668Z namespaces/openshift-kube-scheduler/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.741998441Z namespaces/openshift-kube-scheduler/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742041322Z namespaces/openshift-kube-scheduler/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742069283Z namespaces/openshift-kube-scheduler/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742402529Z namespaces/openshift-kube-scheduler/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74245374Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742550842Z namespaces/openshift-kube-scheduler/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742623823Z namespaces/openshift-kube-scheduler/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742680095Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742700375Z namespaces/openshift-kube-scheduler/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742763586Z namespaces/openshift-kube-scheduler/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742783326Z namespaces/openshift-kube-scheduler/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742848558Z namespaces/openshift-kube-scheduler/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742856998Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.742891279Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743046342Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743251306Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743389048Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743591812Z namespaces/openshift-kube-scheduler/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743773416Z namespaces/openshift-kube-scheduler/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743816177Z namespaces/openshift-kube-scheduler/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743867188Z namespaces/openshift-kube-scheduler/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.743893588Z namespaces/openshift-kube-scheduler/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74396116Z namespaces/openshift-kube-storage-version-migrator-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74401359Z namespaces/openshift-kube-storage-version-migrator-operator/openshift-kube-storage-version-migrator-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744078062Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744110332Z namespaces/openshift-kube-storage-version-migrator-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744182754Z namespaces/openshift-kube-storage-version-migrator-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744223205Z namespaces/openshift-kube-storage-version-migrator-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744326136Z namespaces/openshift-kube-storage-version-migrator-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744445389Z namespaces/openshift-kube-storage-version-migrator-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744550531Z namespaces/openshift-kube-storage-version-migrator-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744598962Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744640023Z namespaces/openshift-kube-storage-version-migrator-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744694194Z namespaces/openshift-kube-storage-version-migrator-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744731995Z namespaces/openshift-kube-storage-version-migrator-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744816726Z namespaces/openshift-kube-storage-version-migrator-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744861617Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744913528Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.744989529Z namespaces/openshift-kube-storage-version-migrator-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74503897Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745049361Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745084911Z namespaces/openshift-kube-storage-version-migrator-operator/coordination.k8s.io/leases/openshift-kube-storage-version-migrator-operator-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745143723Z namespaces/openshift-kube-storage-version-migrator-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745217214Z namespaces/openshift-kube-storage-version-migrator-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745324976Z namespaces/openshift-kube-storage-version-migrator-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745412448Z namespaces/openshift-kube-storage-version-migrator-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745586681Z namespaces/openshift-kube-storage-version-migrator-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745662173Z namespaces/openshift-kube-storage-version-migrator-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745786695Z namespaces/openshift-kube-storage-version-migrator-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.745913058Z namespaces/openshift-kube-storage-version-migrator-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746087411Z namespaces/openshift-kube-storage-version-migrator-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746137082Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746203093Z namespaces/openshift-kube-storage-version-migrator-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746260204Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746303925Z namespaces/openshift-kube-storage-version-migrator-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746370487Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746421508Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746503699Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746582051Z namespaces/openshift-kube-storage-version-migrator-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746639322Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746685573Z namespaces/openshift-kube-storage-version-migrator-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746745934Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746791635Z namespaces/openshift-kube-storage-version-migrator-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746853266Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746869926Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.746913207Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74706969Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.747279944Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.747425367Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.747618771Z namespaces/openshift-kube-storage-version-migrator-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.747949507Z namespaces/openshift-kube-storage-version-migrator-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.747958667Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-7658d9b749-5npqn/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748000128Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-7658d9b749-5npqn/kube-storage-version-migrator-operator-7658d9b749-5npqn.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748142401Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-7658d9b749-5npqn/kube-storage-version-migrator-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748154671Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-7658d9b749-5npqn/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748159721Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-7658d9b749-5npqn/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748251073Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-7658d9b749-5npqn/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748366455Z namespaces/openshift-kube-storage-version-migrator-operator/pods/kube-storage-version-migrator-operator-7658d9b749-5npqn/kube-storage-version-migrator-operator/kube-storage-version-migrator-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748483398Z namespaces/openshift-kube-storage-version-migrator-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748501318Z namespaces/openshift-kube-storage-version-migrator-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748568179Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74860085Z namespaces/openshift-kube-storage-version-migrator-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748677121Z namespaces/openshift-kube-storage-version-migrator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748720442Z namespaces/openshift-kube-storage-version-migrator/openshift-kube-storage-version-migrator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748813384Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748850625Z namespaces/openshift-kube-storage-version-migrator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748903186Z namespaces/openshift-kube-storage-version-migrator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.748948327Z namespaces/openshift-kube-storage-version-migrator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749030748Z namespaces/openshift-kube-storage-version-migrator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749136001Z namespaces/openshift-kube-storage-version-migrator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749252003Z namespaces/openshift-kube-storage-version-migrator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749298024Z namespaces/openshift-kube-storage-version-migrator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749328714Z namespaces/openshift-kube-storage-version-migrator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749373435Z namespaces/openshift-kube-storage-version-migrator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749423756Z namespaces/openshift-kube-storage-version-migrator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749499988Z namespaces/openshift-kube-storage-version-migrator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749552559Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.74958965Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749673381Z namespaces/openshift-kube-storage-version-migrator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749719812Z namespaces/openshift-kube-storage-version-migrator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749751663Z namespaces/openshift-kube-storage-version-migrator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749860335Z namespaces/openshift-kube-storage-version-migrator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.749941976Z namespaces/openshift-kube-storage-version-migrator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750048988Z namespaces/openshift-kube-storage-version-migrator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75012491Z namespaces/openshift-kube-storage-version-migrator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750263693Z namespaces/openshift-kube-storage-version-migrator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750391195Z namespaces/openshift-kube-storage-version-migrator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750564018Z namespaces/openshift-kube-storage-version-migrator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75061813Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75066744Z namespaces/openshift-kube-storage-version-migrator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750698081Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750751982Z namespaces/openshift-kube-storage-version-migrator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750807213Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750846204Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.750928916Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751008737Z namespaces/openshift-kube-storage-version-migrator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751040458Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751094179Z namespaces/openshift-kube-storage-version-migrator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75113314Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751193101Z namespaces/openshift-kube-storage-version-migrator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751299993Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751307503Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751355174Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751506477Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75168576Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.751830363Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752029177Z namespaces/openshift-kube-storage-version-migrator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752228541Z namespaces/openshift-kube-storage-version-migrator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752245531Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752274592Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/migrator-7cc9c7ffd7-t5p2l.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752356753Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/graceful-termination/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752364964Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/graceful-termination/graceful-termination/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752372484Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/graceful-termination/graceful-termination/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752410554Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/graceful-termination/graceful-termination/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752502296Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/graceful-termination/graceful-termination/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752569118Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/graceful-termination/graceful-termination/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752603708Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/migrator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752610689Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/migrator/migrator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752614589Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/migrator/migrator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752644739Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/migrator/migrator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752744821Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/migrator/migrator/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752812132Z namespaces/openshift-kube-storage-version-migrator/pods/migrator-7cc9c7ffd7-t5p2l/migrator/migrator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752846293Z namespaces/openshift-kube-storage-version-migrator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752885914Z namespaces/openshift-kube-storage-version-migrator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752918024Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.752960275Z namespaces/openshift-kube-storage-version-migrator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753014326Z namespaces/openshift-lws-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753052537Z namespaces/openshift-lws-operator/openshift-lws-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753116508Z namespaces/openshift-lws-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753156049Z namespaces/openshift-lws-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75322453Z namespaces/openshift-lws-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753272811Z namespaces/openshift-lws-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753355083Z namespaces/openshift-lws-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753495586Z namespaces/openshift-lws-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753615748Z namespaces/openshift-lws-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753666189Z namespaces/openshift-lws-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75370611Z namespaces/openshift-lws-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753747Z namespaces/openshift-lws-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753786971Z namespaces/openshift-lws-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753871543Z namespaces/openshift-lws-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753904594Z namespaces/openshift-lws-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.753943875Z namespaces/openshift-lws-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754028166Z namespaces/openshift-lws-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754061247Z namespaces/openshift-lws-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754070517Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754126988Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/b8b2488c.x-k8s.io.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75422026Z namespaces/openshift-lws-operator/coordination.k8s.io/leases/openshift-lws-operator-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754283111Z namespaces/openshift-lws-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754318782Z namespaces/openshift-lws-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754440034Z namespaces/openshift-lws-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754537796Z namespaces/openshift-lws-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754902793Z namespaces/openshift-lws-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.754976185Z namespaces/openshift-lws-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755191929Z namespaces/openshift-lws-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755368062Z namespaces/openshift-lws-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755569386Z namespaces/openshift-lws-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755628987Z namespaces/openshift-lws-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755680498Z namespaces/openshift-lws-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75573389Z namespaces/openshift-lws-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75576804Z namespaces/openshift-lws-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755827221Z namespaces/openshift-lws-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755858412Z namespaces/openshift-lws-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.755946994Z namespaces/openshift-lws-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756028685Z namespaces/openshift-lws-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756083376Z namespaces/openshift-lws-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756126307Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756208519Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75625094Z namespaces/openshift-lws-operator/monitoring.coreos.com/servicemonitors/lws-controller-manager-metrics-monitor.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756314981Z namespaces/openshift-lws-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756345671Z namespaces/openshift-lws-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756404343Z namespaces/openshift-lws-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756414693Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756511705Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756692128Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.756874162Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757018195Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757256779Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757413442Z namespaces/openshift-lws-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757595556Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757633396Z namespaces/openshift-lws-operator/operators.coreos.com/installplans/install-vn4z2.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757720538Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757757799Z namespaces/openshift-lws-operator/operators.coreos.com/operatorconditions/leader-worker-set.v1.0.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75782456Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757856451Z namespaces/openshift-lws-operator/operators.coreos.com/operatorgroups/leader-worker-set.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757926802Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.757945073Z namespaces/openshift-lws-operator/operators.coreos.com/subscriptions/leader-worker-set.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758032384Z namespaces/openshift-lws-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758041635Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758078645Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/lws-controller-manager-86dd6ccfdd-57rh6.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758190977Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758204888Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758209478Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758244608Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758595865Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758668537Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-57rh6/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758723158Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758741588Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/lws-controller-manager-86dd6ccfdd-9mv22.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75884852Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7588607Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/manager/manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758869621Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/manager/manager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.758903921Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/manager/manager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759015983Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/manager/manager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759084135Z namespaces/openshift-lws-operator/pods/lws-controller-manager-86dd6ccfdd-9mv22/manager/manager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759130026Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759173937Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/openshift-lws-operator-fd8ccff4c-7msm6.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75933601Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/openshift-lws-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75934592Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/openshift-lws-operator/openshift-lws-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75935424Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/openshift-lws-operator/openshift-lws-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759388241Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/openshift-lws-operator/openshift-lws-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759559584Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/openshift-lws-operator/openshift-lws-operator/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759635386Z namespaces/openshift-lws-operator/pods/openshift-lws-operator-fd8ccff4c-7msm6/openshift-lws-operator/openshift-lws-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759653766Z namespaces/openshift-lws-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759709657Z namespaces/openshift-lws-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759765088Z namespaces/openshift-lws-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759788289Z namespaces/openshift-lws-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75984203Z namespaces/openshift-machine-api/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75985075Z namespaces/openshift-machine-api/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.75985741Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.759891531Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760046304Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760272168Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760419611Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760642455Z namespaces/openshift-machine-api/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760825129Z namespaces/openshift-machine-config-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760838449Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760845669Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.76086986Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-controller.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.760968022Z namespaces/openshift-machine-config-operator/monitoring.coreos.com/prometheusrules/machine-config-daemon.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761039023Z namespaces/openshift-machine-config-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761051413Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761088854Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761255137Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761443161Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761581064Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761781228Z namespaces/openshift-machine-config-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761973521Z namespaces/openshift-marketplace/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761984101Z namespaces/openshift-marketplace/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.761989252Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762002972Z namespaces/openshift-marketplace/monitoring.coreos.com/prometheusrules/marketplace-alert-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762079154Z namespaces/openshift-marketplace/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762089274Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762107544Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/certified-operators.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762236666Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/community-operators.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762321438Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-marketplace.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.76241576Z namespaces/openshift-marketplace/operators.coreos.com/catalogsources/redhat-operators.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762461161Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762497971Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.762663815Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.76291303Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763051542Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763270057Z namespaces/openshift-marketplace/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763467581Z namespaces/openshift-monitoring/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763492421Z namespaces/openshift-monitoring/openshift-monitoring.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763569852Z namespaces/openshift-monitoring/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763610743Z namespaces/openshift-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763677034Z namespaces/openshift-monitoring/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763709705Z namespaces/openshift-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.763852128Z namespaces/openshift-monitoring/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.764285017Z namespaces/openshift-monitoring/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.764661244Z namespaces/openshift-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.764891998Z namespaces/openshift-monitoring/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.764925659Z namespaces/openshift-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.76497361Z namespaces/openshift-monitoring/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.765023901Z namespaces/openshift-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.765121773Z namespaces/openshift-monitoring/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.765178734Z namespaces/openshift-monitoring/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.765245035Z namespaces/openshift-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.765331487Z namespaces/openshift-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.765381338Z namespaces/openshift-monitoring/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.765415968Z namespaces/openshift-monitoring/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.768611061Z namespaces/openshift-monitoring/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.768753364Z namespaces/openshift-monitoring/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.769827235Z namespaces/openshift-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.769908686Z namespaces/openshift-monitoring/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.770892226Z namespaces/openshift-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771014798Z namespaces/openshift-monitoring/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771504438Z namespaces/openshift-monitoring/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771669631Z namespaces/openshift-monitoring/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771692701Z namespaces/openshift-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771850134Z namespaces/openshift-monitoring/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771875405Z namespaces/openshift-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771930846Z namespaces/openshift-monitoring/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.771982337Z namespaces/openshift-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772063329Z namespaces/openshift-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7721403Z namespaces/openshift-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772186811Z namespaces/openshift-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772254292Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772377905Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772401595Z namespaces/openshift-monitoring/monitoring.coreos.com/alertmanagers/main.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772497077Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772542118Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheuses/k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77265332Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7726724Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/alertmanager-main-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772787412Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/cluster-monitoring-operator-prometheus-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.772958126Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kube-state-metrics-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773046468Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/kubernetes-monitoring-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773316103Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/node-exporter-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773454896Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-prometheus-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773582368Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-k8s-thanos-sidecar-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77368006Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/prometheus-operator-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773785222Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/telemetry.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773877354Z namespaces/openshift-monitoring/monitoring.coreos.com/prometheusrules/thanos-querier.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773958546Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.773983656Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/alertmanager-main.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774086188Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/cluster-monitoring-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77419818Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kube-state-metrics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774276842Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/kubelet.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774391984Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/metrics-server.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774482746Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/node-exporter.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774626679Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/openshift-state-metrics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77471035Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774810442Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/prometheus-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774891124Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/telemeter-client.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.774975735Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-querier.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775063157Z namespaces/openshift-monitoring/monitoring.coreos.com/servicemonitors/thanos-sidecar.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775139249Z namespaces/openshift-monitoring/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775180589Z namespaces/openshift-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775310372Z namespaces/openshift-monitoring/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775319792Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775355263Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775561247Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775777741Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.775930274Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776126178Z namespaces/openshift-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776368422Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776388933Z namespaces/openshift-monitoring/operators.coreos.com/operatorgroups/openshift-cluster-monitoring.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776499355Z namespaces/openshift-monitoring/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776508265Z namespaces/openshift-monitoring/pods/alertmanager-main-0/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776588747Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager-main-0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77675774Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77677245Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776778741Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776786781Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776901523Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776971654Z namespaces/openshift-monitoring/pods/alertmanager-main-0/alertmanager/alertmanager/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776988034Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.776996545Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777001725Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777043776Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777144128Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77726663Z namespaces/openshift-monitoring/pods/alertmanager-main-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777296331Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777314501Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777321861Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777328691Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777444054Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777504705Z namespaces/openshift-monitoring/pods/alertmanager-main-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777532365Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777549596Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777558146Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777601677Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777695639Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77776178Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-metric/kube-rbac-proxy-metric/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77779815Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777807301Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777811561Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777856622Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.777944533Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778015945Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778038935Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778043295Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778049635Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778106696Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778212149Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77828006Z namespaces/openshift-monitoring/pods/alertmanager-main-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77831632Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778324791Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778329131Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778355251Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778475534Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778535265Z namespaces/openshift-monitoring/pods/alertmanager-main-0/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778571886Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778618277Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/cluster-monitoring-operator-76cf8966f8-rd75s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778722939Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/cluster-monitoring-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778730729Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/cluster-monitoring-operator/cluster-monitoring-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778735549Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/cluster-monitoring-operator/cluster-monitoring-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.778758879Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/cluster-monitoring-operator/cluster-monitoring-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.779551055Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.779621966Z namespaces/openshift-monitoring/pods/cluster-monitoring-operator-76cf8966f8-rd75s/cluster-monitoring-operator/cluster-monitoring-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.779660377Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.779707628Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-state-metrics-56c5594674-jlgj7.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77981847Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-main/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77982834Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.77983244Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.779872331Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.779965953Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780052654Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780095065Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-self/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780102206Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780107406Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780186937Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780293679Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78036534Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780406831Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-state-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780413372Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-state-metrics/kube-state-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780417492Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-state-metrics/kube-state-metrics/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780444382Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-state-metrics/kube-state-metrics/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780632616Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-state-metrics/kube-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780699887Z namespaces/openshift-monitoring/pods/kube-state-metrics-56c5594674-jlgj7/kube-state-metrics/kube-state-metrics/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780739648Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780771939Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/metrics-server-76c658bcb6-46bmb.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78087383Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/metrics-server/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780881181Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/metrics-server/metrics-server/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780885541Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/metrics-server/metrics-server/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.780923111Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/metrics-server/metrics-server/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781020064Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/metrics-server/metrics-server/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781088545Z namespaces/openshift-monitoring/pods/metrics-server-76c658bcb6-46bmb/metrics-server/metrics-server/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781121025Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781175327Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/monitoring-plugin-7b7b7bc8bf-97vtd.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781270648Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/monitoring-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781278758Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/monitoring-plugin/monitoring-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781282949Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/monitoring-plugin/monitoring-plugin/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781325219Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/monitoring-plugin/monitoring-plugin/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781415141Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/monitoring-plugin/monitoring-plugin/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781484622Z namespaces/openshift-monitoring/pods/monitoring-plugin-7b7b7bc8bf-97vtd/monitoring-plugin/monitoring-plugin/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781530013Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781572384Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/node-exporter-c4wwp.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781687046Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/init-textfile/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781695626Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/init-textfile/init-textfile/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781699557Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/init-textfile/init-textfile/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781744987Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781821149Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7818861Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781929461Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781936401Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781947411Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.781998932Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782089614Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782157835Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782222117Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/node-exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782237977Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/node-exporter/node-exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782246217Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/node-exporter/node-exporter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782281438Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78240237Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782466772Z namespaces/openshift-monitoring/pods/node-exporter-c4wwp/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782525703Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782575104Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/node-exporter-nqbpx.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782682006Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/init-textfile/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782691266Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/init-textfile/init-textfile/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782696676Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/init-textfile/init-textfile/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782728597Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782807408Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78287108Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78290756Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78291641Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78292085Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.782957981Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783050753Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783124165Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783149535Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/node-exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783159285Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/node-exporter/node-exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783188126Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/node-exporter/node-exporter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783277628Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783379359Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783449101Z namespaces/openshift-monitoring/pods/node-exporter-nqbpx/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783490262Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783518212Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/node-exporter-w6m2w.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783623034Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/init-textfile/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783632704Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/init-textfile/init-textfile/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783637184Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/init-textfile/init-textfile/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783656285Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/init-textfile/init-textfile/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783733006Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/init-textfile/init-textfile/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783807208Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/init-textfile/init-textfile/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783824198Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783832098Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783835928Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78389244Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.783994662Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784065653Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784085473Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/node-exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784092873Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/node-exporter/node-exporter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784103894Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/node-exporter/node-exporter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784144165Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/node-exporter/node-exporter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784282377Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/node-exporter/node-exporter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784350529Z namespaces/openshift-monitoring/pods/node-exporter-w6m2w/node-exporter/node-exporter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784370499Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78443108Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/openshift-state-metrics-79f6958cdf-24hlw.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784566953Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-main/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784575023Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-main/kube-rbac-proxy-main/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784590143Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784651804Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784747946Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784812247Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-main/kube-rbac-proxy-main/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784835298Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-self/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784851368Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-self/kube-rbac-proxy-self/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.784858438Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.7849192Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785001241Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785071943Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/kube-rbac-proxy-self/kube-rbac-proxy-self/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785109703Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/openshift-state-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785129924Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/openshift-state-metrics/openshift-state-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785136124Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/openshift-state-metrics/openshift-state-metrics/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785214665Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/openshift-state-metrics/openshift-state-metrics/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785347598Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/openshift-state-metrics/openshift-state-metrics/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785417069Z namespaces/openshift-monitoring/pods/openshift-state-metrics-79f6958cdf-24hlw/openshift-state-metrics/openshift-state-metrics/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78547612Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785558192Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus-k8s-0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785748766Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785764916Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785776666Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785812487Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.785922829Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78598415Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/config-reloader/config-reloader/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786017101Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786038742Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786049792Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786077812Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786199205Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786255246Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/init-config-reloader/init-config-reloader/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786305357Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786314047Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786318237Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786329107Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786439889Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786516121Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-thanos/kube-rbac-proxy-thanos/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786534491Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786542301Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786547381Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786587422Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786680794Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786750275Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786762705Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786779576Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786795276Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786806826Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.786914639Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78698076Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787020601Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787029161Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787035031Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787070302Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787780076Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787843207Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/prometheus/prometheus/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787900718Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787913428Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787917988Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.787938608Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788046741Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788117552Z namespaces/openshift-monitoring/pods/prometheus-k8s-0/thanos-sidecar/thanos-sidecar/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788151773Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788243494Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/prometheus-operator-5945b5cfd-9jx8b.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788382657Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788404678Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788411638Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788423798Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788570171Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788662993Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788723884Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/prometheus-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788730594Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/prometheus-operator/prometheus-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788737154Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/prometheus-operator/prometheus-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.788812516Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/prometheus-operator/prometheus-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789079271Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/prometheus-operator/prometheus-operator/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789203663Z namespaces/openshift-monitoring/pods/prometheus-operator-5945b5cfd-9jx8b/prometheus-operator/prometheus-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789257294Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789345916Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/prometheus-operator-admission-webhook-85574b5f49-5kt4c.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78952901Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/prometheus-operator-admission-webhook/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78954808Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.78955714Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789615401Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789768154Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789857086Z namespaces/openshift-monitoring/pods/prometheus-operator-admission-webhook-85574b5f49-5kt4c/prometheus-operator-admission-webhook/prometheus-operator-admission-webhook/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.789929267Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790013729Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/telemeter-client-786b8f867b-bmpv5.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790230793Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790248784Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790255314Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790307185Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790458788Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.79055074Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790614411Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/reload/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790624261Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/reload/reload/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790630281Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/reload/reload/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790690292Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/reload/reload/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790815245Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/reload/reload/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790910507Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/reload/reload/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790978518Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/telemeter-client/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790990378Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/telemeter-client/telemeter-client/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.790994898Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/telemeter-client/telemeter-client/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791056229Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/telemeter-client/telemeter-client/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791346805Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/telemeter-client/telemeter-client/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791437237Z namespaces/openshift-monitoring/pods/telemeter-client-786b8f867b-bmpv5/telemeter-client/telemeter-client/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791483538Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.79157651Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/thanos-querier-6597b86945-qg9rk.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791735833Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791745133Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791750163Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791799374Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.791924866Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792013938Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-metrics/kube-rbac-proxy-metrics/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792151121Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-rules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792160891Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-rules/kube-rbac-proxy-rules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792190982Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792244213Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792370135Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792466817Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-rules/kube-rbac-proxy-rules/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792519048Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-web/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792525898Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-web/kube-rbac-proxy-web/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792544598Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792592339Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792712872Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792805044Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy-web/kube-rbac-proxy-web/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792868955Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792890545Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792908186Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.792938036Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793062879Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793183081Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793234732Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/prom-label-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793243102Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/prom-label-proxy/prom-label-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793249582Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/prom-label-proxy/prom-label-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793307724Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/prom-label-proxy/prom-label-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793426526Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/prom-label-proxy/prom-label-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793531178Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/prom-label-proxy/prom-label-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793571409Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/thanos-query/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793582219Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/thanos-query/thanos-query/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793588959Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/thanos-query/thanos-query/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.79363897Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/thanos-query/thanos-query/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793794613Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/thanos-query/thanos-query/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793896985Z namespaces/openshift-monitoring/pods/thanos-querier-6597b86945-qg9rk/thanos-query/thanos-query/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.793943146Z namespaces/openshift-monitoring/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794005447Z namespaces/openshift-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794068738Z namespaces/openshift-monitoring/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794120339Z namespaces/openshift-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794285743Z namespaces/openshift-multus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794345724Z namespaces/openshift-multus/openshift-multus.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794440066Z namespaces/openshift-multus/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794490867Z namespaces/openshift-multus/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794561338Z namespaces/openshift-multus/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794612949Z namespaces/openshift-multus/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.794897484Z namespaces/openshift-multus/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795003766Z namespaces/openshift-multus/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795111489Z namespaces/openshift-multus/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.79519118Z namespaces/openshift-multus/apps/daemonsets/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795246501Z namespaces/openshift-multus/apps/daemonsets/multus-additional-cni-plugins.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795443855Z namespaces/openshift-multus/apps/daemonsets/multus.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795590118Z namespaces/openshift-multus/apps/daemonsets/network-metrics-daemon.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.79567029Z namespaces/openshift-multus/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795730211Z namespaces/openshift-multus/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795788152Z namespaces/openshift-multus/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795842123Z namespaces/openshift-multus/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.795956945Z namespaces/openshift-multus/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.796013676Z namespaces/openshift-multus/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.796065907Z namespaces/openshift-multus/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.79620491Z namespaces/openshift-multus/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.796290992Z namespaces/openshift-multus/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.796340453Z namespaces/openshift-multus/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.796518976Z namespaces/openshift-multus/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.796634438Z namespaces/openshift-multus/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.797494315Z namespaces/openshift-multus/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.797671389Z namespaces/openshift-multus/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.798360872Z namespaces/openshift-multus/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.798491555Z namespaces/openshift-multus/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.798618057Z namespaces/openshift-multus/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.798667658Z namespaces/openshift-multus/core/configmaps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.798710769Z namespaces/openshift-multus/core/configmaps/cni-copy-resources.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.798819051Z namespaces/openshift-multus/core/configmaps/default-cni-sysctl-allowlist.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.798935723Z namespaces/openshift-multus/core/configmaps/multus-daemon-config.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799026025Z namespaces/openshift-multus/core/configmaps/whereabouts-flatfile-config.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799132967Z namespaces/openshift-multus/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799150438Z namespaces/openshift-multus/core/serviceaccounts/metrics-daemon-sa.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799365492Z namespaces/openshift-multus/core/serviceaccounts/multus-ac.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799446773Z namespaces/openshift-multus/core/serviceaccounts/multus-ancillary-tools.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799532585Z namespaces/openshift-multus/core/serviceaccounts/multus.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799588836Z namespaces/openshift-multus/core/services/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799628497Z namespaces/openshift-multus/core/services/network-metrics-service.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799679848Z namespaces/openshift-multus/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799720809Z namespaces/openshift-multus/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.79977822Z namespaces/openshift-multus/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799834331Z namespaces/openshift-multus/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799864712Z namespaces/openshift-multus/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.799927883Z namespaces/openshift-multus/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800009314Z namespaces/openshift-multus/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800125657Z namespaces/openshift-multus/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800193978Z namespaces/openshift-multus/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800251389Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80029727Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800339071Z namespaces/openshift-multus/monitoring.coreos.com/servicemonitors/monitor-network.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800395042Z namespaces/openshift-multus/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800443083Z namespaces/openshift-multus/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800485274Z namespaces/openshift-multus/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800498224Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800574135Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800699678Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.800882491Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801021274Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80131369Z namespaces/openshift-multus/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801622016Z namespaces/openshift-multus/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801633056Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801710437Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/multus-additional-cni-plugins-6zp58.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801889491Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/bond-cni-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801898581Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801904021Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.801965353Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802081845Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802201567Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802267619Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802279099Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/cni-plugins/cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802290209Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80233431Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802469163Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802565314Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802615995Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/egress-router-binary-copy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802624655Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802629635Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802670926Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802789669Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802883171Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802936032Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802946042Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.802993923Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803057894Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803156536Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803270628Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803294659Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/routeoverride-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803316139Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803324899Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80335084Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803447582Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803517663Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803541173Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni-bincopy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803557734Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803581674Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803591984Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803678756Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803739577Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803784918Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803795698Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803799838Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803828049Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803921501Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.803984612Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-6zp58/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804019013Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804067464Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/multus-additional-cni-plugins-qbc85.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804219517Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/bond-cni-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804236167Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804241547Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804262718Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804348849Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80442241Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804454231Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804462341Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/cni-plugins/cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804467391Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804499712Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804584164Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804657725Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804689916Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/egress-router-binary-copy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804697326Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804701986Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804736207Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804823498Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80488465Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804927931Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804936391Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804940611Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.804967101Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805039643Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805102314Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805142485Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/routeoverride-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805149545Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805153545Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805209916Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805329318Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80542598Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805474761Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni-bincopy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805481621Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805491471Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805536742Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805659265Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805754176Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805801557Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805808578Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805815678Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805866959Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.805993261Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806084883Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-qbc85/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806146144Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806235716Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/multus-additional-cni-plugins-ww8gr.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806398149Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/bond-cni-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806407659Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/bond-cni-plugin/bond-cni-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80641506Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/bond-cni-plugin/bond-cni-plugin/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80646549Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/bond-cni-plugin/bond-cni-plugin/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806584513Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/bond-cni-plugin/bond-cni-plugin/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806674874Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/bond-cni-plugin/bond-cni-plugin/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806770887Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806778496Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/cni-plugins/cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806782917Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/cni-plugins/cni-plugins/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.806833808Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/cni-plugins/cni-plugins/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80695251Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/cni-plugins/cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807045532Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/cni-plugins/cni-plugins/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807094693Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/egress-router-binary-copy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807105493Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/egress-router-binary-copy/egress-router-binary-copy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807115413Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/egress-router-binary-copy/egress-router-binary-copy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807186264Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/egress-router-binary-copy/egress-router-binary-copy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807311077Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/egress-router-binary-copy/egress-router-binary-copy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807409229Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/egress-router-binary-copy/egress-router-binary-copy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80745426Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80746175Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80748915Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807525921Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807615493Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807720665Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/kube-multus-additional-cni-plugins/kube-multus-additional-cni-plugins/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807760726Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/routeoverride-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807768306Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/routeoverride-cni/routeoverride-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807772166Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/routeoverride-cni/routeoverride-cni/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.807833867Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/routeoverride-cni/routeoverride-cni/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.8079554Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/routeoverride-cni/routeoverride-cni/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808046341Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/routeoverride-cni/routeoverride-cni/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808097853Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni-bincopy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808106342Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni-bincopy/whereabouts-cni-bincopy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808112423Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808195234Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808409139Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.80850167Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni-bincopy/whereabouts-cni-bincopy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808552551Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808560771Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni/whereabouts-cni/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808564911Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni/whereabouts-cni/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808619592Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni/whereabouts-cni/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808734245Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni/whereabouts-cni/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808826897Z namespaces/openshift-multus/pods/multus-additional-cni-plugins-ww8gr/whereabouts-cni/whereabouts-cni/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808877217Z namespaces/openshift-multus/pods/multus-lrbz6/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.808937459Z namespaces/openshift-multus/pods/multus-lrbz6/multus-lrbz6.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.809050271Z namespaces/openshift-multus/pods/multus-lrbz6/kube-multus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.809058411Z namespaces/openshift-multus/pods/multus-lrbz6/kube-multus/kube-multus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.809062571Z namespaces/openshift-multus/pods/multus-lrbz6/kube-multus/kube-multus/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.809109172Z namespaces/openshift-multus/pods/multus-lrbz6/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810557021Z namespaces/openshift-multus/pods/multus-lrbz6/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810659912Z namespaces/openshift-multus/pods/multus-lrbz6/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810700803Z namespaces/openshift-multus/pods/multus-qnvtm/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810773665Z namespaces/openshift-multus/pods/multus-qnvtm/multus-qnvtm.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810879317Z namespaces/openshift-multus/pods/multus-qnvtm/kube-multus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810887397Z namespaces/openshift-multus/pods/multus-qnvtm/kube-multus/kube-multus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810891487Z namespaces/openshift-multus/pods/multus-qnvtm/kube-multus/kube-multus/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.810941758Z namespaces/openshift-multus/pods/multus-qnvtm/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.811723223Z namespaces/openshift-multus/pods/multus-qnvtm/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.811818185Z namespaces/openshift-multus/pods/multus-qnvtm/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.811865726Z namespaces/openshift-multus/pods/multus-w49lz/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.811927677Z namespaces/openshift-multus/pods/multus-w49lz/multus-w49lz.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.812039449Z namespaces/openshift-multus/pods/multus-w49lz/kube-multus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.812050979Z namespaces/openshift-multus/pods/multus-w49lz/kube-multus/kube-multus/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81205523Z namespaces/openshift-multus/pods/multus-w49lz/kube-multus/kube-multus/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.812107711Z namespaces/openshift-multus/pods/multus-w49lz/kube-multus/kube-multus/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.812826495Z namespaces/openshift-multus/pods/multus-w49lz/kube-multus/kube-multus/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.812929107Z namespaces/openshift-multus/pods/multus-w49lz/kube-multus/kube-multus/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.812969288Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813033209Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/network-metrics-daemon-82lt6.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813144761Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813156061Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813161051Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813240253Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813366915Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813459327Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813504538Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/network-metrics-daemon/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813513448Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813525258Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81359576Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813842854Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813940727Z namespaces/openshift-multus/pods/network-metrics-daemon-82lt6/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.813981147Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814049699Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/network-metrics-daemon-jms2f.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814145471Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814156211Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814162931Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814241602Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814363345Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814481027Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814528898Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/network-metrics-daemon/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814538648Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814546878Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81460522Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814805693Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814904105Z namespaces/openshift-multus/pods/network-metrics-daemon-jms2f/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.814950006Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815011368Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/network-metrics-daemon-mg256.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815113439Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81512144Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/kube-rbac-proxy/kube-rbac-proxy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81512571Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/kube-rbac-proxy/kube-rbac-proxy/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815172671Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/kube-rbac-proxy/kube-rbac-proxy/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815317794Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/kube-rbac-proxy/kube-rbac-proxy/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815411745Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/kube-rbac-proxy/kube-rbac-proxy/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815459896Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/network-metrics-daemon/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815470006Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/network-metrics-daemon/network-metrics-daemon/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815476197Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/network-metrics-daemon/network-metrics-daemon/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815540368Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/network-metrics-daemon/network-metrics-daemon/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815820553Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/network-metrics-daemon/network-metrics-daemon/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815924345Z namespaces/openshift-multus/pods/network-metrics-daemon-mg256/network-metrics-daemon/network-metrics-daemon/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.815969436Z namespaces/openshift-multus/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816032787Z namespaces/openshift-multus/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816091678Z namespaces/openshift-multus/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816099609Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81615717Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/multus-whereabouts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816283732Z namespaces/openshift-multus/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816350234Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816410845Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816527807Z namespaces/openshift-multus/rbac.authorization.k8s.io/roles/whereabouts-cni.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816586598Z namespaces/openshift-multus/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816647579Z namespaces/openshift-multus/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81670637Z namespaces/openshift-must-gather-4rtcj/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816714541Z namespaces/openshift-must-gather-4rtcj/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816719131Z namespaces/openshift-must-gather-4rtcj/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816783442Z namespaces/openshift-must-gather-4rtcj/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.816971126Z namespaces/openshift-must-gather-4rtcj/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81719051Z namespaces/openshift-must-gather-4rtcj/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.817378444Z namespaces/openshift-must-gather-4rtcj/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.817641899Z namespaces/openshift-must-gather-4rtcj/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.817876713Z namespaces/openshift-network-console/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.817930204Z namespaces/openshift-network-console/openshift-network-console.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818008196Z namespaces/openshift-network-console/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818062087Z namespaces/openshift-network-console/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818111068Z namespaces/openshift-network-console/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818182329Z namespaces/openshift-network-console/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818281491Z namespaces/openshift-network-console/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818472075Z namespaces/openshift-network-console/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818596598Z namespaces/openshift-network-console/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818625528Z namespaces/openshift-network-console/apps/deployments/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818675379Z namespaces/openshift-network-console/apps/deployments/networking-console-plugin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81874973Z namespaces/openshift-network-console/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818792521Z namespaces/openshift-network-console/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818833582Z namespaces/openshift-network-console/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818878033Z namespaces/openshift-network-console/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.818964515Z namespaces/openshift-network-console/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819001565Z namespaces/openshift-network-console/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819030296Z namespaces/openshift-network-console/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819122228Z namespaces/openshift-network-console/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819162249Z namespaces/openshift-network-console/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81923128Z namespaces/openshift-network-console/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819346092Z namespaces/openshift-network-console/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819425564Z namespaces/openshift-network-console/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819545396Z namespaces/openshift-network-console/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819629998Z namespaces/openshift-network-console/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.81976987Z namespaces/openshift-network-console/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.819894473Z namespaces/openshift-network-console/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820014195Z namespaces/openshift-network-console/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820069716Z namespaces/openshift-network-console/core/configmaps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820117277Z namespaces/openshift-network-console/core/configmaps/networking-console-plugin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820180269Z namespaces/openshift-network-console/core/services/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820299311Z namespaces/openshift-network-console/core/services/networking-console-plugin.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820355632Z namespaces/openshift-network-console/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820401213Z namespaces/openshift-network-console/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820437334Z namespaces/openshift-network-console/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820486555Z namespaces/openshift-network-console/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820543776Z namespaces/openshift-network-console/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820602077Z namespaces/openshift-network-console/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820711379Z namespaces/openshift-network-console/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820819901Z namespaces/openshift-network-console/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820881492Z namespaces/openshift-network-console/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820938953Z namespaces/openshift-network-console/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.820998044Z namespaces/openshift-network-console/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.821060775Z namespaces/openshift-network-console/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.821117237Z namespaces/openshift-network-console/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.821133037Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.821195548Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.821399492Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.821635237Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82181591Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822072375Z namespaces/openshift-network-console/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822439342Z namespaces/openshift-network-console/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822451873Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822504304Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/networking-console-plugin-7bc7489cb5-6zs8d.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822613256Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/networking-console-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822622886Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/networking-console-plugin/networking-console-plugin/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822628016Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/networking-console-plugin/networking-console-plugin/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822678677Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/networking-console-plugin/networking-console-plugin/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82280671Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/networking-console-plugin/networking-console-plugin/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822901291Z namespaces/openshift-network-console/pods/networking-console-plugin-7bc7489cb5-6zs8d/networking-console-plugin/networking-console-plugin/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.822947472Z namespaces/openshift-network-console/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823006673Z namespaces/openshift-network-console/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823058114Z namespaces/openshift-network-console/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823132086Z namespaces/openshift-network-console/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823215308Z namespaces/openshift-network-diagnostics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823283309Z namespaces/openshift-network-diagnostics/openshift-network-diagnostics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82334865Z namespaces/openshift-network-diagnostics/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823409201Z namespaces/openshift-network-diagnostics/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823466942Z namespaces/openshift-network-diagnostics/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823527314Z namespaces/openshift-network-diagnostics/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823665197Z namespaces/openshift-network-diagnostics/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823793929Z namespaces/openshift-network-diagnostics/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823918291Z namespaces/openshift-network-diagnostics/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.823974232Z namespaces/openshift-network-diagnostics/apps/daemonsets/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824028633Z namespaces/openshift-network-diagnostics/apps/daemonsets/network-check-target.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824109405Z namespaces/openshift-network-diagnostics/apps/deployments/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824187207Z namespaces/openshift-network-diagnostics/apps/deployments/network-check-source.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824299019Z namespaces/openshift-network-diagnostics/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82435204Z namespaces/openshift-network-diagnostics/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824421361Z namespaces/openshift-network-diagnostics/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824457472Z namespaces/openshift-network-diagnostics/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824585175Z namespaces/openshift-network-diagnostics/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824641236Z namespaces/openshift-network-diagnostics/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824702877Z namespaces/openshift-network-diagnostics/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824814479Z namespaces/openshift-network-diagnostics/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82487463Z namespaces/openshift-network-diagnostics/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.824942042Z namespaces/openshift-network-diagnostics/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.825084174Z namespaces/openshift-network-diagnostics/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.825231317Z namespaces/openshift-network-diagnostics/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.825489442Z namespaces/openshift-network-diagnostics/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.825596634Z namespaces/openshift-network-diagnostics/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82587548Z namespaces/openshift-network-diagnostics/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.825991872Z namespaces/openshift-network-diagnostics/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826129565Z namespaces/openshift-network-diagnostics/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826232717Z namespaces/openshift-network-diagnostics/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826297908Z namespaces/openshift-network-diagnostics/core/serviceaccounts/network-diagnostics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826370629Z namespaces/openshift-network-diagnostics/core/services/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826447831Z namespaces/openshift-network-diagnostics/core/services/network-check-source.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826551493Z namespaces/openshift-network-diagnostics/core/services/network-check-target.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826594264Z namespaces/openshift-network-diagnostics/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826661535Z namespaces/openshift-network-diagnostics/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826748117Z namespaces/openshift-network-diagnostics/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826800578Z namespaces/openshift-network-diagnostics/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.826863399Z namespaces/openshift-network-diagnostics/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82690028Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827003952Z namespaces/openshift-network-diagnostics/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827097313Z namespaces/openshift-network-diagnostics/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827158475Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827224436Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827293937Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827343698Z namespaces/openshift-network-diagnostics/monitoring.coreos.com/servicemonitors/network-check-source.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82739955Z namespaces/openshift-network-diagnostics/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.8274523Z namespaces/openshift-network-diagnostics/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827504872Z namespaces/openshift-network-diagnostics/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827511902Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827558073Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.827727126Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82793376Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828093983Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828364468Z namespaces/openshift-network-diagnostics/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828576482Z namespaces/openshift-network-diagnostics/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828584042Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828643694Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/network-check-source-7bc5b6bf85-gvfp6.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828745486Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/check-endpoints/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828759196Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/check-endpoints/check-endpoints/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828767046Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/check-endpoints/check-endpoints/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.828811817Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/check-endpoints/check-endpoints/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82897999Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/check-endpoints/check-endpoints/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829082712Z namespaces/openshift-network-diagnostics/pods/network-check-source-7bc5b6bf85-gvfp6/check-endpoints/check-endpoints/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829126373Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829188494Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/network-check-target-7lgzb.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829301016Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/network-check-target-container/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829310147Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829315227Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829360398Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.82948135Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829583502Z namespaces/openshift-network-diagnostics/pods/network-check-target-7lgzb/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829631333Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829691044Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/network-check-target-9ztdl.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829776056Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/network-check-target-container/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829785216Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829790666Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829841497Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.829958729Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830057921Z namespaces/openshift-network-diagnostics/pods/network-check-target-9ztdl/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830104052Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830189654Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/network-check-target-r6xpf.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830313966Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/network-check-target-container/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830330557Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/network-check-target-container/network-check-target-container/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830340967Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/network-check-target-container/network-check-target-container/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830397758Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/network-check-target-container/network-check-target-container/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830544741Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/network-check-target-container/network-check-target-container/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830631482Z namespaces/openshift-network-diagnostics/pods/network-check-target-r6xpf/network-check-target-container/network-check-target-container/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830708324Z namespaces/openshift-network-diagnostics/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830790246Z namespaces/openshift-network-diagnostics/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830858967Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830868187Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.830912888Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/network-diagnostics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831048291Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831100612Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831160893Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/network-diagnostics.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831324766Z namespaces/openshift-network-diagnostics/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831673823Z namespaces/openshift-network-diagnostics/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831726544Z namespaces/openshift-network-diagnostics/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831788705Z namespaces/openshift-network-node-identity/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831848776Z namespaces/openshift-network-node-identity/openshift-network-node-identity.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831929388Z namespaces/openshift-network-node-identity/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.831980009Z namespaces/openshift-network-node-identity/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.83204294Z namespaces/openshift-network-node-identity/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832097601Z namespaces/openshift-network-node-identity/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832231324Z namespaces/openshift-network-node-identity/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832338876Z namespaces/openshift-network-node-identity/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832444918Z namespaces/openshift-network-node-identity/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832502199Z namespaces/openshift-network-node-identity/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.83256272Z namespaces/openshift-network-node-identity/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832618651Z namespaces/openshift-network-node-identity/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832674862Z namespaces/openshift-network-node-identity/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832785345Z namespaces/openshift-network-node-identity/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832840006Z namespaces/openshift-network-node-identity/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.832894947Z namespaces/openshift-network-node-identity/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833001419Z namespaces/openshift-network-node-identity/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833057Z namespaces/openshift-network-node-identity/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.83306456Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833114981Z namespaces/openshift-network-node-identity/coordination.k8s.io/leases/ovnkube-identity.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833182682Z namespaces/openshift-network-node-identity/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833249224Z namespaces/openshift-network-node-identity/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833377936Z namespaces/openshift-network-node-identity/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833482478Z namespaces/openshift-network-node-identity/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833612301Z namespaces/openshift-network-node-identity/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833721363Z namespaces/openshift-network-node-identity/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.833834205Z namespaces/openshift-network-node-identity/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834017809Z namespaces/openshift-network-node-identity/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834157091Z namespaces/openshift-network-node-identity/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834241023Z namespaces/openshift-network-node-identity/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834307934Z namespaces/openshift-network-node-identity/core/serviceaccounts/network-node-identity.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834372726Z namespaces/openshift-network-node-identity/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834439897Z namespaces/openshift-network-node-identity/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834503908Z namespaces/openshift-network-node-identity/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834559489Z namespaces/openshift-network-node-identity/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.83462478Z namespaces/openshift-network-node-identity/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834679112Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834789274Z namespaces/openshift-network-node-identity/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834894866Z namespaces/openshift-network-node-identity/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.834953457Z namespaces/openshift-network-node-identity/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835009458Z namespaces/openshift-network-node-identity/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835068589Z namespaces/openshift-network-node-identity/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.83512658Z namespaces/openshift-network-node-identity/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835208812Z namespaces/openshift-network-node-identity/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835219482Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835272663Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835459467Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835676581Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.835851935Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836096049Z namespaces/openshift-network-node-identity/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836421416Z namespaces/openshift-network-node-identity/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836476277Z namespaces/openshift-network-node-identity/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836541688Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836549248Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836610789Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/rolebindings/network-node-identity-leases.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836670351Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836730322Z namespaces/openshift-network-node-identity/rbac.authorization.k8s.io/roles/network-node-identity-leases.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836792153Z namespaces/openshift-network-node-identity/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836849444Z namespaces/openshift-network-node-identity/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836909375Z namespaces/openshift-network-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.836975566Z namespaces/openshift-network-operator/openshift-network-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837051038Z namespaces/openshift-network-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837112789Z namespaces/openshift-network-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837190101Z namespaces/openshift-network-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837266422Z namespaces/openshift-network-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837390745Z namespaces/openshift-network-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837518407Z namespaces/openshift-network-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.83763633Z namespaces/openshift-network-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.83768325Z namespaces/openshift-network-operator/apps/daemonsets/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837749352Z namespaces/openshift-network-operator/apps/daemonsets/iptables-alerter.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837828273Z namespaces/openshift-network-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837885244Z namespaces/openshift-network-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.837950765Z namespaces/openshift-network-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838000246Z namespaces/openshift-network-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838104379Z namespaces/openshift-network-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.8381862Z namespaces/openshift-network-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838261822Z namespaces/openshift-network-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838371314Z namespaces/openshift-network-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838442645Z namespaces/openshift-network-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838452295Z namespaces/openshift-network-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838507087Z namespaces/openshift-network-operator/coordination.k8s.io/leases/network-operator-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838575908Z namespaces/openshift-network-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838630739Z namespaces/openshift-network-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838802392Z namespaces/openshift-network-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.838916315Z namespaces/openshift-network-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.839128769Z namespaces/openshift-network-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.839265071Z namespaces/openshift-network-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.839482995Z namespaces/openshift-network-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.839596258Z namespaces/openshift-network-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.839767651Z namespaces/openshift-network-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.839839562Z namespaces/openshift-network-operator/core/configmaps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.839902624Z namespaces/openshift-network-operator/core/configmaps/applied-cluster.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840028756Z namespaces/openshift-network-operator/core/configmaps/iptables-alerter-script.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840116308Z namespaces/openshift-network-operator/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840187089Z namespaces/openshift-network-operator/core/serviceaccounts/iptables-alerter.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840272461Z namespaces/openshift-network-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840327492Z namespaces/openshift-network-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840405373Z namespaces/openshift-network-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840444744Z namespaces/openshift-network-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840510526Z namespaces/openshift-network-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840563647Z namespaces/openshift-network-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840674819Z namespaces/openshift-network-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840770501Z namespaces/openshift-network-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840865122Z namespaces/openshift-network-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840923044Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.840993965Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841046846Z namespaces/openshift-network-operator/monitoring.coreos.com/prometheusrules/openshift-network-operator-ipsec-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841124287Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841181559Z namespaces/openshift-network-operator/monitoring.coreos.com/servicemonitors/network-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84127319Z namespaces/openshift-network-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841329751Z namespaces/openshift-network-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841387443Z namespaces/openshift-network-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841395183Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841445994Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.841825341Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842045205Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842242129Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842512365Z namespaces/openshift-network-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842742489Z namespaces/openshift-network-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842751549Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84279984Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/iptables-alerter-hv4xk.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842895622Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/iptables-alerter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842903542Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842908042Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.842962443Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843080366Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843181688Z namespaces/openshift-network-operator/pods/iptables-alerter-hv4xk/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843226288Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84327116Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/iptables-alerter-w6hbk.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843356671Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/iptables-alerter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843364831Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843368911Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843423663Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843513744Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843583936Z namespaces/openshift-network-operator/pods/iptables-alerter-w6hbk/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843602026Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843660977Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/iptables-alerter-x8j28.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843729229Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/iptables-alerter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843738499Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/iptables-alerter/iptables-alerter/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843743419Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/iptables-alerter/iptables-alerter/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843751579Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/iptables-alerter/iptables-alerter/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843845091Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/iptables-alerter/iptables-alerter/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843919992Z namespaces/openshift-network-operator/pods/iptables-alerter-x8j28/iptables-alerter/iptables-alerter/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843932653Z namespaces/openshift-network-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.843989134Z namespaces/openshift-network-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844034464Z namespaces/openshift-network-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844079865Z namespaces/openshift-network-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844113746Z namespaces/openshift-node/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844121386Z namespaces/openshift-node/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844126756Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844181417Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844416762Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844631556Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844744258Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.844958073Z namespaces/openshift-node/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845143026Z namespaces/openshift-operator-lifecycle-manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845189027Z namespaces/openshift-operator-lifecycle-manager/openshift-operator-lifecycle-manager.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845277869Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84531862Z namespaces/openshift-operator-lifecycle-manager/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84537153Z namespaces/openshift-operator-lifecycle-manager/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845414481Z namespaces/openshift-operator-lifecycle-manager/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845506753Z namespaces/openshift-operator-lifecycle-manager/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845578254Z namespaces/openshift-operator-lifecycle-manager/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845660006Z namespaces/openshift-operator-lifecycle-manager/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845706287Z namespaces/openshift-operator-lifecycle-manager/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845743858Z namespaces/openshift-operator-lifecycle-manager/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845783198Z namespaces/openshift-operator-lifecycle-manager/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845814589Z namespaces/openshift-operator-lifecycle-manager/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845903281Z namespaces/openshift-operator-lifecycle-manager/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845933861Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.845984242Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846064964Z namespaces/openshift-operator-lifecycle-manager/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846101445Z namespaces/openshift-operator-lifecycle-manager/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846150616Z namespaces/openshift-operator-lifecycle-manager/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846281338Z namespaces/openshift-operator-lifecycle-manager/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84635867Z namespaces/openshift-operator-lifecycle-manager/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846477812Z namespaces/openshift-operator-lifecycle-manager/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846553533Z namespaces/openshift-operator-lifecycle-manager/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846644935Z namespaces/openshift-operator-lifecycle-manager/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846771788Z namespaces/openshift-operator-lifecycle-manager/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.846970232Z namespaces/openshift-operator-lifecycle-manager/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847033073Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847085144Z namespaces/openshift-operator-lifecycle-manager/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847118575Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847192046Z namespaces/openshift-operator-lifecycle-manager/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847250077Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847294238Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.8473738Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847453411Z namespaces/openshift-operator-lifecycle-manager/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847500302Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847537373Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847584864Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847638025Z namespaces/openshift-operator-lifecycle-manager/monitoring.coreos.com/prometheusrules/olm-alert-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847717576Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847766667Z namespaces/openshift-operator-lifecycle-manager/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847832809Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847842849Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.847867899Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.848025092Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.848244457Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.848382939Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.848615884Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.848810168Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.848839778Z namespaces/openshift-operator-lifecycle-manager/operators.coreos.com/operatorgroups/olm-operators.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84890698Z namespaces/openshift-operator-lifecycle-manager/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84893368Z namespaces/openshift-operator-lifecycle-manager/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.848996661Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849033002Z namespaces/openshift-operator-lifecycle-manager/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849084333Z namespaces/openshift-operators/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849091333Z namespaces/openshift-operators/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849095413Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849131274Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849322378Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849504081Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.849636284Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.84995009Z namespaces/openshift-operators/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850144554Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850189465Z namespaces/openshift-operators/operators.coreos.com/operatorgroups/global-operators.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850253386Z namespaces/openshift-ovn-kubernetes/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850310437Z namespaces/openshift-ovn-kubernetes/openshift-ovn-kubernetes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850365038Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850423129Z namespaces/openshift-ovn-kubernetes/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85045652Z namespaces/openshift-ovn-kubernetes/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850516361Z namespaces/openshift-ovn-kubernetes/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850701535Z namespaces/openshift-ovn-kubernetes/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850780736Z namespaces/openshift-ovn-kubernetes/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850856448Z namespaces/openshift-ovn-kubernetes/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850900558Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.850941489Z namespaces/openshift-ovn-kubernetes/apps/daemonsets/ovnkube-node.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851076032Z namespaces/openshift-ovn-kubernetes/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851095752Z namespaces/openshift-ovn-kubernetes/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851155643Z namespaces/openshift-ovn-kubernetes/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851234155Z namespaces/openshift-ovn-kubernetes/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851316687Z namespaces/openshift-ovn-kubernetes/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851354898Z namespaces/openshift-ovn-kubernetes/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851392448Z namespaces/openshift-ovn-kubernetes/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85148136Z namespaces/openshift-ovn-kubernetes/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85150633Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85151181Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851567102Z namespaces/openshift-ovn-kubernetes/coordination.k8s.io/leases/ovn-kubernetes-master.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851616363Z namespaces/openshift-ovn-kubernetes/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851651073Z namespaces/openshift-ovn-kubernetes/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851853227Z namespaces/openshift-ovn-kubernetes/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.851941709Z namespaces/openshift-ovn-kubernetes/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.852292726Z namespaces/openshift-ovn-kubernetes/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.852421238Z namespaces/openshift-ovn-kubernetes/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.852887057Z namespaces/openshift-ovn-kubernetes/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.852970539Z namespaces/openshift-ovn-kubernetes/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853159273Z namespaces/openshift-ovn-kubernetes/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853241794Z namespaces/openshift-ovn-kubernetes/core/configmaps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853283835Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-config.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853364507Z namespaces/openshift-ovn-kubernetes/core/configmaps/ovnkube-script-lib.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853487469Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85353309Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-control-plane.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853619362Z namespaces/openshift-ovn-kubernetes/core/serviceaccounts/ovn-kubernetes-node.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853665223Z namespaces/openshift-ovn-kubernetes/core/services/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853704693Z namespaces/openshift-ovn-kubernetes/core/services/ovn-kubernetes-node.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853765425Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853788195Z namespaces/openshift-ovn-kubernetes/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853870427Z namespaces/openshift-ovn-kubernetes/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853920537Z namespaces/openshift-ovn-kubernetes/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853952128Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.853965008Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854003299Z namespaces/openshift-ovn-kubernetes/k8s.cni.cncf.io/network-attachment-definitions/default.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85405397Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854086491Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854197573Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854286064Z namespaces/openshift-ovn-kubernetes/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854326395Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854373956Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854437298Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854488068Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/prometheusrules/networking-rules.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85457121Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854605431Z namespaces/openshift-ovn-kubernetes/monitoring.coreos.com/servicemonitors/monitor-ovn-node.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854650982Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854662362Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854705853Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/ovn.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854803515Z namespaces/openshift-ovn-kubernetes/network.operator.openshift.io/operatorpkis/signer.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854827545Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854894036Z namespaces/openshift-ovn-kubernetes/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854937577Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854952568Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.854966518Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.855157872Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.855379906Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85556824Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.856402786Z namespaces/openshift-ovn-kubernetes/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85658899Z namespaces/openshift-ovn-kubernetes/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85660552Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.856656461Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovnkube-node-8h6hb.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.856858585Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-node/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.856873065Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.856878085Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.856891126Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857019578Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857089119Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85713159Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857157251Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857181731Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857202132Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857325094Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857376915Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857424906Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/nbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857433916Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/nbdb/nbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857460297Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/nbdb/nbdb/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857468157Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/nbdb/nbdb/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857546288Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85763367Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85764559Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/northd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85765004Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/northd/northd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85765399Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/northd/northd/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857716072Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/northd/northd/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857815794Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857878985Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/northd/northd/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857921576Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-acl-logging/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857938576Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857961916Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.857979977Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.858052828Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-acl-logging/ovn-acl-logging/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85812655Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85815018Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85815731Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-controller/ovn-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85816153Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.858237862Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.858506837Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.858574088Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.858593129Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovnkube-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.858599179Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.85867601Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.858691141Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860518317Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860569907Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860622899Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/sbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860638459Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/sbdb/sbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860643899Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/sbdb/sbdb/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860653809Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/sbdb/sbdb/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860759991Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860825872Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-8h6hb/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860852113Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.860931545Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovnkube-node-h4csm.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861082828Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-node/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861115838Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861123028Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861136349Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861294732Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861372523Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861405454Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861412354Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861416604Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861434654Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861561307Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861625298Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861674019Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/nbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861686969Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/nbdb/nbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861691089Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/nbdb/nbdb/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861700389Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/nbdb/nbdb/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861800911Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861873053Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861907173Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/northd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861916664Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/northd/northd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861920994Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/northd/northd/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.861942714Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/northd/northd/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862053226Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862118638Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/northd/northd/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862185399Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-acl-logging/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862201489Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862206129Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86222753Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862324322Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862388743Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862397123Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-controller/ovn-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862401093Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862434134Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862809351Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862877252Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862908013Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovnkube-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862920933Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862925253Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.862954604Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865153897Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865248109Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865272129Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/sbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86528581Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/sbdb/sbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86529128Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/sbdb/sbdb/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86532269Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/sbdb/sbdb/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865424622Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865495924Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-h4csm/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865517044Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865590256Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovnkube-node-kg8lz.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865745799Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-node/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865754869Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-node/kube-rbac-proxy-node/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865760609Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865796489Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865923952Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.865995573Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-node/kube-rbac-proxy-node/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866013964Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866027144Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866035434Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866058434Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866232688Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866302349Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/kube-rbac-proxy-ovn-metrics/kube-rbac-proxy-ovn-metrics/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86633128Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/nbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86633914Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/nbdb/nbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86634386Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/nbdb/nbdb/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866379451Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/nbdb/nbdb/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866490223Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/nbdb/nbdb/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866567884Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/nbdb/nbdb/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866585675Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/northd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866590245Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/northd/northd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866595275Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/northd/northd/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866637406Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/northd/northd/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866763688Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/northd/northd/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86682136Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/northd/northd/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.8668625Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-acl-logging/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86687003Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-acl-logging/ovn-acl-logging/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.86687465Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-acl-logging/ovn-acl-logging/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.866903861Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-acl-logging/ovn-acl-logging/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867000153Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-acl-logging/ovn-acl-logging/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867061904Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-acl-logging/ovn-acl-logging/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867102065Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867112015Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-controller/ovn-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867116095Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-controller/ovn-controller/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867142666Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-controller/ovn-controller/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867483042Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-controller/ovn-controller/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867547604Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovn-controller/ovn-controller/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867575124Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovnkube-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867579394Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovnkube-controller/ovnkube-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867583284Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovnkube-controller/ovnkube-controller/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.867628075Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovnkube-controller/ovnkube-controller/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869547903Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovnkube-controller/ovnkube-controller/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869619844Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/ovnkube-controller/ovnkube-controller/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869655415Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/sbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869664505Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/sbdb/sbdb/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869669505Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/sbdb/sbdb/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869707956Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/sbdb/sbdb/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869802978Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/sbdb/sbdb/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869877949Z namespaces/openshift-ovn-kubernetes/pods/ovnkube-node-kg8lz/sbdb/sbdb/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869895189Z namespaces/openshift-ovn-kubernetes/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869952841Z namespaces/openshift-ovn-kubernetes/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.869993702Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870002472Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870043412Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870146545Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/openshift-ovn-kubernetes-nodes-identity-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870297557Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/rolebindings/prometheus-k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870346238Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870394899Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-control-plane-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870482641Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/openshift-ovn-kubernetes-node-limited.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870568393Z namespaces/openshift-ovn-kubernetes/rbac.authorization.k8s.io/roles/prometheus-k8s.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870601323Z namespaces/openshift-ovn-kubernetes/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870639464Z namespaces/openshift-ovn-kubernetes/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870710105Z namespaces/openshift-route-controller-manager/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870722116Z namespaces/openshift-route-controller-manager/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870726116Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870764286Z namespaces/openshift-route-controller-manager/coordination.k8s.io/leases/openshift-route-controllers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870816998Z namespaces/openshift-route-controller-manager/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870824648Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.870855998Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.871110433Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.871340558Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.871486551Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.871706275Z namespaces/openshift-route-controller-manager/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.871934059Z namespaces/openshift-service-ca-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87198454Z namespaces/openshift-service-ca-operator/openshift-service-ca-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872042502Z namespaces/openshift-service-ca-operator/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872081952Z namespaces/openshift-service-ca-operator/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872121093Z namespaces/openshift-service-ca-operator/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872190004Z namespaces/openshift-service-ca-operator/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872296056Z namespaces/openshift-service-ca-operator/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872419019Z namespaces/openshift-service-ca-operator/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872513971Z namespaces/openshift-service-ca-operator/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872558452Z namespaces/openshift-service-ca-operator/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872601653Z namespaces/openshift-service-ca-operator/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872647043Z namespaces/openshift-service-ca-operator/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872695224Z namespaces/openshift-service-ca-operator/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872767926Z namespaces/openshift-service-ca-operator/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872816507Z namespaces/openshift-service-ca-operator/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872856838Z namespaces/openshift-service-ca-operator/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.872931739Z namespaces/openshift-service-ca-operator/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873126323Z namespaces/openshift-service-ca-operator/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873140013Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873145673Z namespaces/openshift-service-ca-operator/coordination.k8s.io/leases/service-ca-operator-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873150483Z namespaces/openshift-service-ca-operator/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873155953Z namespaces/openshift-service-ca-operator/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873259595Z namespaces/openshift-service-ca-operator/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873343577Z namespaces/openshift-service-ca-operator/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873540581Z namespaces/openshift-service-ca-operator/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873619962Z namespaces/openshift-service-ca-operator/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873732764Z namespaces/openshift-service-ca-operator/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.873867837Z namespaces/openshift-service-ca-operator/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874048581Z namespaces/openshift-service-ca-operator/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874088791Z namespaces/openshift-service-ca-operator/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874138023Z namespaces/openshift-service-ca-operator/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874279075Z namespaces/openshift-service-ca-operator/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874325416Z namespaces/openshift-service-ca-operator/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874422538Z namespaces/openshift-service-ca-operator/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874439258Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87451701Z namespaces/openshift-service-ca-operator/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874598241Z namespaces/openshift-service-ca-operator/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874650133Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874670743Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874738674Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874806325Z namespaces/openshift-service-ca-operator/monitoring.coreos.com/servicemonitors/service-ca-operator.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.874823946Z namespaces/openshift-service-ca-operator/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875099821Z namespaces/openshift-service-ca-operator/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875112651Z namespaces/openshift-service-ca-operator/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875118612Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875124242Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875138302Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875326886Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875468069Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875660722Z namespaces/openshift-service-ca-operator/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875849786Z namespaces/openshift-service-ca-operator/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875861956Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5bcbc94c78-dmmwd/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875872546Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5bcbc94c78-dmmwd/service-ca-operator-5bcbc94c78-dmmwd.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875966688Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5bcbc94c78-dmmwd/service-ca-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875976058Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5bcbc94c78-dmmwd/service-ca-operator/service-ca-operator/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875982419Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5bcbc94c78-dmmwd/service-ca-operator/service-ca-operator/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.875996409Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5bcbc94c78-dmmwd/service-ca-operator/service-ca-operator/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876154242Z namespaces/openshift-service-ca-operator/pods/service-ca-operator-5bcbc94c78-dmmwd/service-ca-operator/service-ca-operator/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876360646Z namespaces/openshift-service-ca-operator/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876400337Z namespaces/openshift-service-ca-operator/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876459958Z namespaces/openshift-service-ca-operator/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876493218Z namespaces/openshift-service-ca-operator/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876538029Z namespaces/openshift-service-ca/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87658728Z namespaces/openshift-service-ca/openshift-service-ca.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876642561Z namespaces/openshift-service-ca/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876679872Z namespaces/openshift-service-ca/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876747363Z namespaces/openshift-service-ca/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876781454Z namespaces/openshift-service-ca/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876859116Z namespaces/openshift-service-ca/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.876959138Z namespaces/openshift-service-ca/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87705596Z namespaces/openshift-service-ca/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87709896Z namespaces/openshift-service-ca/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877142461Z namespaces/openshift-service-ca/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877215343Z namespaces/openshift-service-ca/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877259124Z namespaces/openshift-service-ca/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877348755Z namespaces/openshift-service-ca/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877410956Z namespaces/openshift-service-ca/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877449367Z namespaces/openshift-service-ca/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877535909Z namespaces/openshift-service-ca/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87758333Z namespaces/openshift-service-ca/coordination.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87759462Z namespaces/openshift-service-ca/coordination.k8s.io/leases/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877629121Z namespaces/openshift-service-ca/coordination.k8s.io/leases/service-ca-controller-lock.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877675632Z namespaces/openshift-service-ca/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877723562Z namespaces/openshift-service-ca/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877833565Z namespaces/openshift-service-ca/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.877915666Z namespaces/openshift-service-ca/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878019678Z namespaces/openshift-service-ca/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87809487Z namespaces/openshift-service-ca/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878237723Z namespaces/openshift-service-ca/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878396246Z namespaces/openshift-service-ca/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878557959Z namespaces/openshift-service-ca/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.87860951Z namespaces/openshift-service-ca/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878662751Z namespaces/openshift-service-ca/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878700201Z namespaces/openshift-service-ca/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878735832Z namespaces/openshift-service-ca/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878787703Z namespaces/openshift-service-ca/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878839624Z namespaces/openshift-service-ca/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878921596Z namespaces/openshift-service-ca/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.878996297Z namespaces/openshift-service-ca/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879046008Z namespaces/openshift-service-ca/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879085999Z namespaces/openshift-service-ca/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.8791288Z namespaces/openshift-service-ca/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879187261Z namespaces/openshift-service-ca/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879299673Z namespaces/openshift-service-ca/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879317624Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879336214Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879555488Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879673241Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.879843674Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880027687Z namespaces/openshift-service-ca/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880215971Z namespaces/openshift-service-ca/pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880230631Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880278803Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/service-ca-58598695f7-l5j47.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880352484Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/service-ca-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880361034Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/service-ca-controller/service-ca-controller/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880367924Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/service-ca-controller/service-ca-controller/logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880408755Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/service-ca-controller/service-ca-controller/logs/current.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880628889Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/service-ca-controller/service-ca-controller/logs/previous.insecure.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880700641Z namespaces/openshift-service-ca/pods/service-ca-58598695f7-l5j47/service-ca-controller/service-ca-controller/logs/previous.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880733301Z namespaces/openshift-service-ca/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880802793Z namespaces/openshift-service-ca/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880815063Z namespaces/openshift-service-ca/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880849263Z namespaces/openshift-service-ca/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880900335Z namespaces/openshift-user-workload-monitoring/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880939705Z namespaces/openshift-user-workload-monitoring/openshift-user-workload-monitoring.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.880997886Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881037457Z namespaces/openshift-user-workload-monitoring/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881087958Z namespaces/openshift-user-workload-monitoring/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881110629Z namespaces/openshift-user-workload-monitoring/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881234721Z namespaces/openshift-user-workload-monitoring/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881330463Z namespaces/openshift-user-workload-monitoring/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881407754Z namespaces/openshift-user-workload-monitoring/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881457966Z namespaces/openshift-user-workload-monitoring/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881478876Z namespaces/openshift-user-workload-monitoring/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881546497Z namespaces/openshift-user-workload-monitoring/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881559728Z namespaces/openshift-user-workload-monitoring/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881659739Z namespaces/openshift-user-workload-monitoring/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88170367Z namespaces/openshift-user-workload-monitoring/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881743401Z namespaces/openshift-user-workload-monitoring/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881825353Z namespaces/openshift-user-workload-monitoring/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881868394Z namespaces/openshift-user-workload-monitoring/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.881919724Z namespaces/openshift-user-workload-monitoring/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882009506Z namespaces/openshift-user-workload-monitoring/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882115268Z namespaces/openshift-user-workload-monitoring/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88218607Z namespaces/openshift-user-workload-monitoring/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882353263Z namespaces/openshift-user-workload-monitoring/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882560667Z namespaces/openshift-user-workload-monitoring/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882679399Z namespaces/openshift-user-workload-monitoring/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882884543Z namespaces/openshift-user-workload-monitoring/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882913804Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.882988355Z namespaces/openshift-user-workload-monitoring/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883043446Z namespaces/openshift-user-workload-monitoring/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883077257Z namespaces/openshift-user-workload-monitoring/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883083117Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883138388Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88325019Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883355212Z namespaces/openshift-user-workload-monitoring/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883385143Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883430964Z namespaces/openshift-user-workload-monitoring/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883481055Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883526456Z namespaces/openshift-user-workload-monitoring/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883567927Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883575697Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883612748Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88376Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.883937974Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884072046Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884313221Z namespaces/openshift-user-workload-monitoring/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884503885Z namespaces/openshift-user-workload-monitoring/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884531615Z namespaces/openshift-user-workload-monitoring/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884589217Z namespaces/openshift-user-workload-monitoring/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884633128Z namespaces/openshift-user-workload-monitoring/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884681729Z namespaces/openshift/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884727599Z namespaces/openshift/openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88477942Z namespaces/openshift/apps.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884806201Z namespaces/openshift/apps.openshift.io/deploymentconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884858572Z namespaces/openshift/apps/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884898613Z namespaces/openshift/apps/daemonsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.884974314Z namespaces/openshift/apps/deployments.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885058286Z namespaces/openshift/apps/replicasets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885135897Z namespaces/openshift/apps/statefulsets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885181528Z namespaces/openshift/autoscaling/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88525839Z namespaces/openshift/autoscaling/horizontalpodautoscalers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885310661Z namespaces/openshift/batch/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885359442Z namespaces/openshift/batch/cronjobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885433783Z namespaces/openshift/batch/jobs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885474584Z namespaces/openshift/build.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885511995Z namespaces/openshift/build.openshift.io/buildconfigs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885591726Z namespaces/openshift/build.openshift.io/builds.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885645097Z namespaces/openshift/core/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885674288Z namespaces/openshift/core/configmaps.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88577428Z namespaces/openshift/core/endpoints.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885850451Z namespaces/openshift/core/events.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.885929053Z namespaces/openshift/core/persistentvolumeclaims.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886007734Z namespaces/openshift/core/pods.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886089466Z namespaces/openshift/core/replicationcontrollers.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886221748Z namespaces/openshift/core/secrets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886400222Z namespaces/openshift/core/services.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886450053Z namespaces/openshift/discovery.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886501144Z namespaces/openshift/discovery.k8s.io/endpointslices.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886545365Z namespaces/openshift/image.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.886578275Z namespaces/openshift/image.openshift.io/imagestreams.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88786392Z namespaces/openshift/image.openshift.io/imagestreams/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.887911292Z namespaces/openshift/image.openshift.io/imagestreams/cli-artifacts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.887995293Z namespaces/openshift/image.openshift.io/imagestreams/cli.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.888081135Z namespaces/openshift/image.openshift.io/imagestreams/dotnet-runtime.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.888204907Z namespaces/openshift/image.openshift.io/imagestreams/dotnet.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88833188Z namespaces/openshift/image.openshift.io/imagestreams/driver-toolkit.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.888421642Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift-java11.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.888586755Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-eap-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.888740118Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88886445Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-java11-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.888958182Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift-jdk11.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.889051714Z namespaces/openshift/image.openshift.io/imagestreams/fuse7-karaf-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.889184046Z namespaces/openshift/image.openshift.io/imagestreams/golang.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88934346Z namespaces/openshift/image.openshift.io/imagestreams/httpd.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.889456662Z namespaces/openshift/image.openshift.io/imagestreams/installer-artifacts.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.889558414Z namespaces/openshift/image.openshift.io/imagestreams/installer.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.889645185Z namespaces/openshift/image.openshift.io/imagestreams/java-runtime.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.889755237Z namespaces/openshift/image.openshift.io/imagestreams/java.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.88987167Z namespaces/openshift/image.openshift.io/imagestreams/jboss-datagrid73-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.889986472Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890085904Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp3-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890196616Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890364399Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap-xp4-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890477342Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890576284Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk11-runtime-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890676515Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890774327Z namespaces/openshift/image.openshift.io/imagestreams/jboss-eap74-openjdk8-runtime-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890871569Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk11-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.890971511Z namespaces/openshift/image.openshift.io/imagestreams/jboss-webserver57-openjdk8-tomcat9-openshift-ubi8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891079233Z namespaces/openshift/image.openshift.io/imagestreams/jenkins-agent-base.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891180015Z namespaces/openshift/image.openshift.io/imagestreams/jenkins.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891332778Z namespaces/openshift/image.openshift.io/imagestreams/mariadb.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89143122Z namespaces/openshift/image.openshift.io/imagestreams/must-gather.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891530612Z namespaces/openshift/image.openshift.io/imagestreams/mysql.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891634454Z namespaces/openshift/image.openshift.io/imagestreams/network-tools.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891723166Z namespaces/openshift/image.openshift.io/imagestreams/nginx.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891844248Z namespaces/openshift/image.openshift.io/imagestreams/nodejs.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.891989711Z namespaces/openshift/image.openshift.io/imagestreams/oauth-proxy.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.892077353Z namespaces/openshift/image.openshift.io/imagestreams/openjdk-11-rhel7.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.892216096Z namespaces/openshift/image.openshift.io/imagestreams/perl.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.892326538Z namespaces/openshift/image.openshift.io/imagestreams/php.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89244037Z namespaces/openshift/image.openshift.io/imagestreams/postgresql.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.892568223Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso75-openshift-rhel8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.892658854Z namespaces/openshift/image.openshift.io/imagestreams/postgresql13-for-sso76-openshift-rhel8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.892749166Z namespaces/openshift/image.openshift.io/imagestreams/python.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.892877378Z namespaces/openshift/image.openshift.io/imagestreams/redhat-openjdk18-openshift.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893008461Z namespaces/openshift/image.openshift.io/imagestreams/redis.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893114313Z namespaces/openshift/image.openshift.io/imagestreams/ruby.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893238445Z namespaces/openshift/image.openshift.io/imagestreams/sso75-openshift-rhel8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893329037Z namespaces/openshift/image.openshift.io/imagestreams/sso76-openshift-rhel8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893424749Z namespaces/openshift/image.openshift.io/imagestreams/tests.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893513201Z namespaces/openshift/image.openshift.io/imagestreams/tools.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893636563Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11-runtime.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893756726Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-11.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.893880498Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17-runtime.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894015741Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-17.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894132983Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21-runtime.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894246155Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-21.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894336707Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8-runtime.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89447265Z namespaces/openshift/image.openshift.io/imagestreams/ubi8-openjdk-8.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894550791Z namespaces/openshift/k8s.ovn.org/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894594192Z namespaces/openshift/k8s.ovn.org/egressfirewalls.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894681184Z namespaces/openshift/k8s.ovn.org/egressqoses.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894756445Z namespaces/openshift/k8s.ovn.org/userdefinednetworks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894790146Z namespaces/openshift/monitoring.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894842547Z namespaces/openshift/monitoring.coreos.com/servicemonitors.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894873608Z namespaces/openshift/networking.k8s.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894928958Z namespaces/openshift/networking.k8s.io/networkpolicies.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.894974439Z namespaces/openshift/operators.coreos.com/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89499173Z namespaces/openshift/operators.coreos.com/clusterserviceversions/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89500282Z namespaces/openshift/operators.coreos.com/clusterserviceversions/authorino-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.895190574Z namespaces/openshift/operators.coreos.com/clusterserviceversions/cert-manager-operator.v1.20.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.895369407Z namespaces/openshift/operators.coreos.com/clusterserviceversions/dns-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89550537Z namespaces/openshift/operators.coreos.com/clusterserviceversions/limitador-operator.v1.4.0.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.895698184Z namespaces/openshift/operators.coreos.com/clusterserviceversions/rhcl-operator.v1.4.1.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.895878927Z namespaces/openshift/policy/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.895922498Z namespaces/openshift/policy/poddisruptionbudgets.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.895964089Z namespaces/openshift/route.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89599715Z namespaces/openshift/route.openshift.io/routes.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89604898Z namespaces/openshift/template.openshift.io/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896058751Z namespaces/openshift/template.openshift.io/templates/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896097981Z namespaces/openshift/template.openshift.io/templates/cache-service.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896223254Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896346446Z namespaces/openshift/template.openshift.io/templates/cakephp-mysql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896461968Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896568301Z namespaces/openshift/template.openshift.io/templates/dancer-mysql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896725674Z namespaces/openshift/template.openshift.io/templates/datagrid-service.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896831166Z namespaces/openshift/template.openshift.io/templates/django-psql-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.896945628Z namespaces/openshift/template.openshift.io/templates/django-psql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897130522Z namespaces/openshift/template.openshift.io/templates/eap-xp3-basic-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897271655Z namespaces/openshift/template.openshift.io/templates/eap-xp4-basic-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897383707Z namespaces/openshift/template.openshift.io/templates/eap74-basic-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897486999Z namespaces/openshift/template.openshift.io/templates/eap74-https-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897615311Z namespaces/openshift/template.openshift.io/templates/eap74-sso-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897755894Z namespaces/openshift/template.openshift.io/templates/httpd-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897861976Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral-monitored.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.897984198Z namespaces/openshift/template.openshift.io/templates/jenkins-ephemeral.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89808983Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent-monitored.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.898279524Z namespaces/openshift/template.openshift.io/templates/jenkins-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.898409126Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.898514769Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk11-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.898627721Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-basic-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.898723733Z namespaces/openshift/template.openshift.io/templates/jws57-openjdk8-tomcat9-ubi8-https-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.898840765Z namespaces/openshift/template.openshift.io/templates/mariadb-ephemeral.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.898948277Z namespaces/openshift/template.openshift.io/templates/mariadb-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899049109Z namespaces/openshift/template.openshift.io/templates/mysql-ephemeral.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899151441Z namespaces/openshift/template.openshift.io/templates/mysql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899273694Z namespaces/openshift/template.openshift.io/templates/nginx-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899376085Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899489418Z namespaces/openshift/template.openshift.io/templates/nodejs-postgresql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.89959778Z namespaces/openshift/template.openshift.io/templates/openjdk-web-basic-s2i.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899700492Z namespaces/openshift/template.openshift.io/templates/postgresql-ephemeral.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899802414Z namespaces/openshift/template.openshift.io/templates/postgresql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.899897636Z namespaces/openshift/template.openshift.io/templates/rails-pgsql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900022548Z namespaces/openshift/template.openshift.io/templates/rails-postgresql-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900122Z namespaces/openshift/template.openshift.io/templates/react-web-app-example.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900250122Z namespaces/openshift/template.openshift.io/templates/redis-ephemeral.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900348155Z namespaces/openshift/template.openshift.io/templates/redis-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900446616Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-rest-3scale.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900582099Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel-xml.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900692971Z namespaces/openshift/template.openshift.io/templates/s2i-fuse712-spring-boot-2-camel.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900791143Z namespaces/openshift/template.openshift.io/templates/sso75-https.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.900912406Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-https.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.901025408Z namespaces/openshift/template.openshift.io/templates/sso75-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.90114837Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.901294103Z namespaces/openshift/template.openshift.io/templates/sso75-postgresql.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.901427705Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-https.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.901578748Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.901705411Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-postgresql.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.901832504Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-https.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.901947766Z namespaces/openshift/template.openshift.io/templates/sso76-ocp4-x509-postgresql-persistent.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902096859Z network_logs/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902135959Z network_logs/cluster_scale [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902243381Z network_logs/ippools.whereabouts.cni.cncf.io [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902316403Z network_logs/multi-networkpolicy [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902399525Z network_logs/net-attach-def [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902493777Z network_logs/overlappingrangeipreservations.whereabouts.cni.cncf.io [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902636629Z network_logs/ovn_kubernetes_top_pods [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.902729021Z network_logs/ovnk_database_store.tar.gz [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.905150698Z nodes/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.90521213Z nodes/debug [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.905303221Z nodes/ip-10-0-133-160.ec2.internal/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.905347642Z nodes/ip-10-0-133-160.ec2.internal/cpu_affinities.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.905635768Z nodes/ip-10-0-133-160.ec2.internal/dmesg [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.905897393Z nodes/ip-10-0-133-160.ec2.internal/ethtool_channels [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.905981065Z nodes/ip-10-0-133-160.ec2.internal/ethtool_features [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.906059136Z nodes/ip-10-0-133-160.ec2.internal/ip-10-0-133-160.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.906611897Z nodes/ip-10-0-133-160.ec2.internal/irq_affinities.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.90675695Z nodes/ip-10-0-133-160.ec2.internal/lscpu [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.906886052Z nodes/ip-10-0-133-160.ec2.internal/lspci [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.906978264Z nodes/ip-10-0-133-160.ec2.internal/podresources.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.907086766Z nodes/ip-10-0-133-160.ec2.internal/pods_info.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.907220339Z nodes/ip-10-0-133-160.ec2.internal/proc_cmdline [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.90730483Z nodes/ip-10-0-133-160.ec2.internal/sysinfo.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.908969263Z nodes/ip-10-0-133-160.ec2.internal/sysinfo.tgz [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.909115426Z nodes/ip-10-0-137-140.ec2.internal/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.909157667Z nodes/ip-10-0-137-140.ec2.internal/cpu_affinities.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.909500323Z nodes/ip-10-0-137-140.ec2.internal/dmesg [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.909748768Z nodes/ip-10-0-137-140.ec2.internal/ethtool_channels [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.90982591Z nodes/ip-10-0-137-140.ec2.internal/ethtool_features [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.909905721Z nodes/ip-10-0-137-140.ec2.internal/ip-10-0-137-140.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.910387891Z nodes/ip-10-0-137-140.ec2.internal/irq_affinities.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.910532963Z nodes/ip-10-0-137-140.ec2.internal/lscpu [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.910653536Z nodes/ip-10-0-137-140.ec2.internal/lspci [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.910735507Z nodes/ip-10-0-137-140.ec2.internal/podresources.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.91084128Z nodes/ip-10-0-137-140.ec2.internal/pods_info.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.910962432Z nodes/ip-10-0-137-140.ec2.internal/proc_cmdline [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.911044084Z nodes/ip-10-0-137-140.ec2.internal/sysinfo.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.912748407Z nodes/ip-10-0-137-140.ec2.internal/sysinfo.tgz [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.91289502Z nodes/ip-10-0-138-196.ec2.internal/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.91291786Z nodes/ip-10-0-138-196.ec2.internal/cpu_affinities.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.913189445Z nodes/ip-10-0-138-196.ec2.internal/dmesg [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.913492701Z nodes/ip-10-0-138-196.ec2.internal/ethtool_channels [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.913570603Z nodes/ip-10-0-138-196.ec2.internal/ethtool_features [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.913653235Z nodes/ip-10-0-138-196.ec2.internal/ip-10-0-138-196.ec2.internal_logs_kubelet.gz [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.914296177Z nodes/ip-10-0-138-196.ec2.internal/irq_affinities.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.91445916Z nodes/ip-10-0-138-196.ec2.internal/lscpu [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.914588553Z nodes/ip-10-0-138-196.ec2.internal/lspci [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.914668424Z nodes/ip-10-0-138-196.ec2.internal/podresources.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.914782426Z nodes/ip-10-0-138-196.ec2.internal/pods_info.json [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.914889588Z nodes/ip-10-0-138-196.ec2.internal/proc_cmdline [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.9149724Z nodes/ip-10-0-138-196.ec2.internal/sysinfo.log [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.916697144Z nodes/ip-10-0-138-196.ec2.internal/sysinfo.tgz [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.916835006Z pod_network_connectivity_check/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.916869457Z pod_network_connectivity_check/podnetworkconnectivitychecks.yaml [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.916975229Z static-pods/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.91698542Z static-pods/kube-apiserver/ [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.924471956Z [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.924490026Z sent 70,983 bytes received 7,994,887 bytes 16,131,740.00 bytes/sec [must-gather] [must-gather-xsljk] OUT 2026-07-20T11:27:01.924498636Z total size is 95,247,125 speedup is 11.81 [must-gather] [must-gather ] OUT 2026-07-20T11:27:02.243785713Z namespace/openshift-must-gather-4rtcj deleted [must-gather] [must-gather] [must-gather] Reprinting Cluster State: [must-gather] When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: [must-gather] ClusterID: 18243c48-a001-4376-9d1d-4ce350941cb0 [must-gather] ClientVersion: 4.21.10 [must-gather] ClusterVersion: Stable at "4.21.25" [must-gather] ClusterOperators: [must-gather] clusteroperator/authentication is missing [must-gather] clusteroperator/cloud-credential is missing [must-gather] clusteroperator/cluster-autoscaler is missing [must-gather] clusteroperator/config-operator is missing [must-gather] clusteroperator/etcd is missing [must-gather] clusteroperator/machine-api is missing [must-gather] clusteroperator/machine-approver is missing [must-gather] clusteroperator/machine-config is missing [must-gather] clusteroperator/marketplace is missing [must-gather] [must-gather] [git-push-artifacts] WORK_DIR: /workspace/odh-ci-artifacts [git-push-artifacts] REPO_PATH: opendatahub-io/odh-build-metadata [git-push-artifacts] REPO_BRANCH: ci-artifacts [git-push-artifacts] SPARSE_FILE_PATH: test-artifacts/docs [git-push-artifacts] SOURCE_PATH: /workspace/artifacts-dir [git-push-artifacts] DEST_PATH: test-artifacts/kserve-group-test-wc8nk [git-push-artifacts] ALWAYS_PASS: false [git-push-artifacts] configuring gh token [git-push-artifacts] taking github token from Konflux bot [git-push-artifacts] Initialized empty Git repository in /workspace/odh-ci-artifacts/.git/ [git-push-artifacts] Using partial fetch with sparse checkout for: test-artifacts/docs [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] * [new branch] ci-artifacts -> origin/ci-artifacts [git-push-artifacts] Already on 'ci-artifacts' [git-push-artifacts] branch 'ci-artifacts' set up to track 'origin/ci-artifacts'. [git-push-artifacts] TASK_NAME=kserve-group-test-wc8nk-e2e-llm-inference-service [git-push-artifacts] PIPELINERUN_NAME=kserve-group-test-wc8nk [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] -rw-r--r--. 1 root 1001540000 15360953 Jul 20 11:28 /workspace/odh-ci-artifacts/test-artifacts/kserve-group-test-wc8nk/e2e-llm-inference-service.tar.gz [git-push-artifacts] [ci-artifacts ce534d3] Updating CI Artifacts in e2e-llm-inference-service [git-push-artifacts] 1 file changed, 0 insertions(+), 0 deletions(-) [git-push-artifacts] create mode 100644 test-artifacts/kserve-group-test-wc8nk/e2e-llm-inference-service.tar.gz [git-push-artifacts] From https://github.com/opendatahub-io/odh-build-metadata [git-push-artifacts] * branch ci-artifacts -> FETCH_HEAD [git-push-artifacts] Already up to date. [git-push-artifacts] To https://github.com/opendatahub-io/odh-build-metadata.git [git-push-artifacts] 0e425ed..ce534d3 ci-artifacts -> ci-artifacts [fail-if-needed] Failing pipeline because deploy-and-e2e step failed container step-fail-if-needed has failed : [{"key":"StartedAt","value":"2026-07-20T11:28:35.062Z","type":3}]