Time Namespace Component RelatedObject Reason Message

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5 to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-rmgs4

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-rmgs4 to ip-10-0-133-6.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-9dda20

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-9dda20: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-491fe7

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-491fe7: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-5d459a

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-5d459a: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-d1e2ab

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-d1e2ab: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-36ccd3

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-36ccd3: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w1-90051a

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w1-90051a: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-af788a

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-af788a: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-c5bec0

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-c5bec0: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-10f834

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-10f834: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-kkx5w

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-kkx5w to ip-10-0-132-184.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-cd8696ff5-4976v

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-cd8696ff5-4976v to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-skdkh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-skdkh to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-skdxg

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-skdxg to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-swnrt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-swnrt to ip-10-0-133-6.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-549fcdc555-45lrf

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-549fcdc555-45lrf to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

ai-gateway-controller-5d8dddb676-rw8nw

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-rw8nw to ip-10-0-133-6.us-west-1.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-vxhx8

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-vxhx8 to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-86fmg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-86fmg to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-vtsx4

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-vtsx4 to ip-10-0-133-6.us-west-1.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq to ip-10-0-133-6.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-13190c

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-13190c: MaasTenantConfig is not available to restore revocation operands

models-as-a-service

payload-processing-external-model-86b659c8d5-hjmv6

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-86b659c8d5-hjmv6 to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-w6hx4

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-w6hx4 to ip-10-0-133-235.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-549fcdc555-dzzld

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-549fcdc555-dzzld to ip-10-0-132-184.us-west-1.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-pqqxp

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-pqqxp to ip-10-0-132-184.us-west-1.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-5rhg8

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-5rhg8 to ip-10-0-132-184.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-549fcdc555-t4pn8 to ip-10-0-133-235.us-west-1.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-ftccw

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-ftccw to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-7d699c9f7ctz9j4

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7ctz9j4 to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5 to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

ai-gateway-controller-d69f556c7-qx7mw

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-d69f556c7-qx7mw to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-zt9qt

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-zt9qt to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v to ip-10-0-133-235.us-west-1.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-f6dp2

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-f6dp2 to ip-10-0-133-6.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-859b946898-skctn

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-859b946898-skctn to ip-10-0-132-184.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-7cf9557897-mngcx

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7cf9557897-mngcx to ip-10-0-132-184.us-west-1.compute.internal

kuadrant-system

authorino-56c6c9ddd5-4ldrz

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-4ldrz to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

router-default-66bfccb89c-84k2x

FailedScheduling

no nodes available to schedule pods

openshift-ingress

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9 to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst to ip-10-0-132-184.us-west-1.compute.internal

kuadrant-system

authorino-57f7f8659f-8tcrc

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-8tcrc to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

router-default-66bfccb89c-84k2x

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-66bfccb89c-84k2x

Scheduled

Successfully assigned openshift-ingress/router-default-66bfccb89c-84k2x to ip-10-0-132-184.us-west-1.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-cmqzr

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-cmqzr to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-xgbfh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xgbfh to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6 to ip-10-0-133-235.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-141891

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-141891: MaasTenantConfig is not available to restore revocation operands

llm

echo-simulator-kserve-64b6f8f96c-zg7ps

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-zg7ps to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-f4mfz

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-f4mfz to ip-10-0-132-184.us-west-1.compute.internal

kuadrant-system

authorino-648d4dbc8d-djsnr

Scheduled

Successfully assigned kuadrant-system/authorino-648d4dbc8d-djsnr to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp to ip-10-0-132-184.us-west-1.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-tt4m7 to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9 to ip-10-0-133-235.us-west-1.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

test-curl-7388-12987b

Scheduled

Successfully assigned opendatahub/test-curl-7388-12987b to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-7jh5t

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7jh5t to ip-10-0-132-184.us-west-1.compute.internal

kuadrant-system

authorino-6f6944bcdd-k4vqp

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-k4vqp to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mpwlf to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m to ip-10-0-132-184.us-west-1.compute.internal

openshift-jobset-operator

jobset-controller-manager-7578997f76-wwszp

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-7578997f76-wwszp to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm to ip-10-0-132-184.us-west-1.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-s9qz2

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-s9qz2 to ip-10-0-133-6.us-west-1.compute.internal

kuadrant-system

authorino-85bcdc76cd-x98xg

Scheduled

Successfully assigned kuadrant-system/authorino-85bcdc76cd-x98xg to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-mmdl5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mmdl5 to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-491fe7-openshift-default-7774566777-6957l to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6 to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94 to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9 to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8 to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8 to ip-10-0-133-6.us-west-1.compute.internal

kuadrant-system

authorino-874888cd5-l4bhw

Scheduled

Successfully assigned kuadrant-system/authorino-874888cd5-l4bhw to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-zsxt6

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zsxt6 to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-hx8xk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-hx8xk to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-hj5bm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-hj5bm to ip-10-0-132-184.us-west-1.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-xzvqn

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-xzvqn to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-82ql9

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-82ql9 to ip-10-0-133-235.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-cc668

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-cc668 to ip-10-0-133-235.us-west-1.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-dcl2t

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-dcl2t to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

maas-controller-8456fc546f-85nz4

Scheduled

Successfully assigned opendatahub/maas-controller-8456fc546f-85nz4 to ip-10-0-132-184.us-west-1.compute.internal

opendatahub

maas-controller-868857dbd9-6b89z

Scheduled

Successfully assigned opendatahub/maas-controller-868857dbd9-6b89z to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

model-serving-api-75bd89fdfc-w5ktz

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-w5ktz to ip-10-0-133-6.us-west-1.compute.internal

opendatahub

odh-model-controller-769875db56-zm64c

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-zm64c to ip-10-0-132-184.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-6ffc4dfcb5-7l6vp

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6ffc4dfcb5-7l6vp to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk to ip-10-0-132-184.us-west-1.compute.internal

opendatahub

test-curl-7379-d99e5f

Scheduled

Successfully assigned opendatahub/test-curl-7379-d99e5f to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-blsp9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-blsp9 to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd to ip-10-0-132-184.us-west-1.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-24x8l

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-24x8l to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29 to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4 to ip-10-0-133-6.us-west-1.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-57xgd

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-57xgd to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh to ip-10-0-132-184.us-west-1.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5 to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-5d77786fb9-9gmhr

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5d77786fb9-9gmhr to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-6s4br

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6s4br to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-5s8q9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-5s8q9 to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4 to ip-10-0-133-235.us-west-1.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-wsld5

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-wsld5 to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-42zq8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-42zq8 to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-2nvtc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-2nvtc to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-6dcf757899-26fdx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-26fdx to ip-10-0-133-235.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-sm9xv

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-sm9xv to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-5d77786fb9-j8wzv

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5d77786fb9-j8wzv to ip-10-0-133-235.us-west-1.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-kscfp

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-kscfp to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt to ip-10-0-132-184.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-s7xrk

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-s7xrk to ip-10-0-133-235.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g to ip-10-0-132-184.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

payload-processing-7bb9489676-vzvct

Scheduled

Successfully assigned openshift-ingress/payload-processing-7bb9489676-vzvct to ip-10-0-133-6.us-west-1.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-66bfccb89c from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-66bfccb89c

SuccessfulCreate

Created pod: router-default-66bfccb89c-84k2x
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-66bfccb89c-84k2x

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-66bfccb89c-84k2x

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-66bfccb89c-84k2x

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-66bfccb89c-84k2x

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-66bfccb89c-84k2x

AddedInterface

Add eth0 [10.132.0.18/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-66bfccb89c-84k2x

Created

Container created

openshift-ingress

kubelet

router-default-66bfccb89c-84k2x

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

router-default-66bfccb89c-84k2x

Started

Container started

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-f4mfz

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-f4mfz

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-f4mfz

AddedInterface

Add eth0 [10.132.0.25/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-f4mfz

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-f4mfz

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 24.915s (24.915s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-f4mfz

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-f4mfz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-f4mfz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-6dcf757899 from 0 to 1

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-58fd47455f from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-mpwlf

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-58fd47455f

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-mpwlf

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.879s (9.879s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk

Created

Container created

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.744s (9.744s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-58fd47455f-5r9gk

Started

Container started
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-549fcdc555

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-549fcdc555-t4pn8

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-549fcdc555

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-549fcdc555-45lrf

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-549fcdc555 from 0 to 3

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-549fcdc555

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-549fcdc555-dzzld

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-549fcdc555-45lrf

AddedInterface

Add eth0 [10.134.0.19/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-549fcdc555-t4pn8

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

NeedsReinstall

calculated deployment install is bad

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-549fcdc555-dzzld

AddedInterface

Add eth0 [10.132.0.26/23] from ovn-kubernetes
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.383s (3.383s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.566s (3.566s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Started

Container started

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.316s (3.316s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-s9qz2

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-s9qz2

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 849ms (849ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-45lrf

Started

Container started
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-s9qz2

AddedInterface

Add eth0 [10.134.0.20/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Created

Container created

opendatahub-operator-system

opendatahub-operator-controller-manager-549fcdc555-45lrf_3680755b-6784-498a-8cc9-6d586599da2f

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-549fcdc555-45lrf_3680755b-6784-498a-8cc9-6d586599da2f became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-t4pn8

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 866ms (866ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Created

Container created

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-549fcdc555-dzzld

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 889ms (889ms including waiting). Image size: 273709574 bytes.
(x2)

opendatahub-operator-system

operator-lifecycle-manager

install-tgt2q

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-s9qz2

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 3.803s (3.803s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-s9qz2

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-s9qz2

Started

Container started

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-s9qz2_8c278562-fde9-418d-92e4-c488e0f1a0e7 became leader
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-sm9xv

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-sm9xv

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-sm9xv

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-57xgd

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-57xgd

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.
(x8)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-57xgd

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-vxhx8

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing
(x2)

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreateFailed

Failed to create Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator: Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": service "cert-manager-webhook" not found

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-vxhx8

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-vxhx8

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-ingress

multus

kube-auth-proxy-5d77786fb9-j8wzv

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

replicaset-controller

kube-auth-proxy-5d77786fb9

SuccessfulCreate

Created pod: kube-auth-proxy-5d77786fb9-j8wzv

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-sm9xv

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-sm9xv

Started

Container started

openshift-ingress

multus

kube-auth-proxy-5d77786fb9-9gmhr

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

kube-auth-proxy-5d77786fb9

SuccessfulCreate

Created pod: kube-auth-proxy-5d77786fb9-9gmhr

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-5d77786fb9 from 0 to 2

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-9gmhr

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-sm9xv

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 5.237s (5.237s including waiting). Image size: 424244516 bytes.

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-7d699c9f7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-7d699c9f7ctz9j4

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-7d699c9f7c from 0 to 1

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-57xgd

Created

Container created

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-7d699c9f7ctz9j4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-7d699c9f7ctz9j4

Created

Container created

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-7d699c9f7ctz9j4

Started

Container started

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-7d699c9f7ctz9j4

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-57xgd

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 2.641s (2.641s including waiting). Image size: 190804149 bytes.

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-dcl2t

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-57xgd

Started

Container started

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-sm9xv_f3623722-4af1-4497-b340-45d3cc4ed5d4

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-sm9xv_f3623722-4af1-4497-b340-45d3cc4ed5d4 became leader

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-dcl2t

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-j8wzv

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

kuadrant-system

dns-operator-controller-manager-8557cbdb84-57xgd_614e5e7e-aa5c-4329-97ec-71ad619fe4a1

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-57xgd_614e5e7e-aa5c-4329-97ec-71ad619fe4a1 became leader

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-dcl2t

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-j8wzv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.983s (2.983s including waiting). Image size: 188904677 bytes.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-xzvqn

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-5rhg8

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-9gmhr

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.759s (3.759s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-9gmhr

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-9gmhr

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-j8wzv

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-5d77786fb9-j8wzv

Started

Container started

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-5rhg8

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-vxhx8

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5rhg8

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

multus

authorino-operator-7b7bb4bc77-xzvqn

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-xzvqn

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-vxhx8

Created

Container created

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-vxhx8

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 5.386s (5.386s including waiting). Image size: 309650901 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-dcl2t

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 3.849s (3.849s including waiting). Image size: 177967631 bytes.
(x3)

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreateFailed

Failed to create Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator: Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": no endpoints available for service "cert-manager-webhook"

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-dcl2t

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-dcl2t

Started

Container started

kuadrant-system

limitador-operator-controller-manager-5778bb4449-dcl2t_16de2d9a-c39b-4fa7-92a6-25236309b8a5

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-dcl2t_16de2d9a-c39b-4fa7-92a6-25236309b8a5 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-xzvqn

Started

Container started

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-xzvqn

Created

Container created
(x12)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

authorino-operator-7b7bb4bc77-xzvqn_4101c96d-e3e0-4849-8b38-333d3cabf49d

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-xzvqn_4101c96d-e3e0-4849-8b38-333d3cabf49d became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-xzvqn

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 2.617s (2.617s including waiting). Image size: 167738465 bytes.

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5rhg8

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 4.213s (4.213s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5rhg8

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5rhg8

Created

Container created

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-s7xrk

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-s7xrk

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-s7xrk

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-s7xrk

Created

Container created

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-s7xrk

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-wsld5

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-wsld5

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

multus

limitador-limitador-74dfcfb894-wsld5

AddedInterface

Add eth0 [10.132.0.28/23] from ovn-kubernetes

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-kkx5w

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

cert-manager

multus

cert-manager-597b95cb84-kkx5w

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-kkx5w

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

cert-manager

kubelet

cert-manager-597b95cb84-kkx5w

Started

Container started

cert-manager

kubelet

cert-manager-597b95cb84-kkx5w

Created

Container created

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-wsld5

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-wsld5

Started

Container started

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-wsld5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 3.381s (3.381s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-sm9xv

Killing

Stopping container manager

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-sm9xv

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-f6dp2

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-f6dp2

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-f6dp2

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-f6dp2

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-f6dp2

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-f6dp2

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 1.851s (1.851s including waiting). Image size: 195729000 bytes.

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-s7xrk_781976df-7294-49d5-bf82-eee0c3a26867

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-s7xrk_781976df-7294-49d5-bf82-eee0c3a26867 became leader

opendatahub

kserve-module-controller-manager-7566cf5fc8-f6dp2_ba718a4d-6212-41e5-b12f-cdcbc876741b

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-f6dp2_ba718a4d-6212-41e5-b12f-cdcbc876741b became leader
(x3)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

multus

limitador-limitador-7c9b87d574-24x8l

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-24x8l

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-kscfp

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-kscfp

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-kscfp

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-8tcrc

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-24x8l

Pulled

Container image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-24x8l

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-24x8l

Started

Container started

kuadrant-system

multus

authorino-57f7f8659f-8tcrc

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-57f7f8659f-8tcrc

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-4ldrz

kuadrant-system

multus

authorino-56c6c9ddd5-4ldrz

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-56c6c9ddd5-4ldrz

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-57f7f8659f-8tcrc

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.054s (3.054s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-57f7f8659f-8tcrc

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-8tcrc

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-4ldrz

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-kscfp

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-kscfp

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-kscfp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 5.568s (5.568s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-4ldrz

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.299s (2.299s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-4ldrz

Created

Container created

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-8tcrc

kuadrant-system

kubelet

authorino-57f7f8659f-8tcrc

Killing

Stopping container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-wsld5

Killing

Stopping container limitador

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-wsld5

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-7jh5t

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-7jh5t

Started

Container started

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-7jh5t

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-7jh5t

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-7jh5t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-7jh5t

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-7jh5t

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-f4mfz

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-f4mfz
(x6)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-zt9qt

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-cc668

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-w5ktz

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

kubelet

model-serving-api-75bd89fdfc-w5ktz

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

kubelet

kserve-controller-manager-776cc66d47-zt9qt

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

kubelet

odh-model-controller-769875db56-zm64c

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-cc668

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

odh-model-controller-769875db56-zm64c

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-cc668

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-zm64c

opendatahub

multus

model-serving-api-75bd89fdfc-w5ktz

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

opendatahub

multus

kserve-controller-manager-776cc66d47-zt9qt

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-cc668

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 1.993s (1.993s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-cc668

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-w5ktz

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.32s (2.32s including waiting). Image size: 184925761 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-cc668

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-w5ktz

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-zt9qt

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.458s (3.458s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-zt9qt

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-zt9qt

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-w5ktz

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-zm64c

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-zm64c

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-zm64c

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.355s (3.355s including waiting). Image size: 236755567 bytes.

opendatahub

llmisvc-controller-manager-869d5dcc5d-cc668_52d497d3-7583-4b7b-ba47-144ebbc89337

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-cc668_52d497d3-7583-4b7b-ba47-144ebbc89337 became leader

opendatahub

kserve-controller-manager-776cc66d47-zt9qt_5787c979-4144-484b-8abf-b9cf5ab97b19

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-zt9qt_5787c979-4144-484b-8abf-b9cf5ab97b19 became leader

opendatahub

odh-model-controller-769875db56-zm64c_24d8feab-c94c-48e0-b520-6995a3a6784d

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-zm64c_24d8feab-c94c-48e0-b520-6995a3a6784d became leader

opendatahub

replicaset-controller

maas-controller-8456fc546f

SuccessfulCreate

Created pod: maas-controller-8456fc546f-85nz4

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-8456fc546f from 0 to 1

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-868857dbd9 from 0 to 1

opendatahub

multus

maas-controller-8456fc546f-85nz4

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-8456fc546f-85nz4

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:60ae2524f867a210b42428cccf9ebde1c56d305cf88f0980b4ae7231fe33386e"

opendatahub

replicaset-controller

maas-controller-868857dbd9

SuccessfulCreate

Created pod: maas-controller-868857dbd9-6b89z

opendatahub

multus

maas-controller-868857dbd9-6b89z

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-868857dbd9-6b89z

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:60ae2524f867a210b42428cccf9ebde1c56d305cf88f0980b4ae7231fe33386e"

opendatahub

kubelet

maas-controller-868857dbd9-6b89z

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:60ae2524f867a210b42428cccf9ebde1c56d305cf88f0980b4ae7231fe33386e" in 7.427s (7.427s including waiting). Image size: 200103442 bytes.

opendatahub

kubelet

maas-controller-868857dbd9-6b89z

Created

Container created

opendatahub

kubelet

maas-controller-868857dbd9-6b89z

Started

Container started

opendatahub

kubelet

maas-controller-8456fc546f-85nz4

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:60ae2524f867a210b42428cccf9ebde1c56d305cf88f0980b4ae7231fe33386e" in 8.489s (8.489s including waiting). Image size: 200103442 bytes.

opendatahub

kubelet

maas-controller-8456fc546f-85nz4

Started

Container started

opendatahub

kubelet

maas-controller-8456fc546f-85nz4

Created

Container created

opendatahub

maas-controller-868857dbd9-6b89z_7006ec5f-9527-46ee-b08c-6615ff6ea2d3

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-868857dbd9-6b89z_7006ec5f-9527-46ee-b08c-6615ff6ea2d3 became leader

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

FailedMount

MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-vqxvh"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-m5cln"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-7578997f76

SuccessfulCreate

Created pod: jobset-controller-manager-7578997f76-wwszp

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-7578997f76 from 0 to 1

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

opendatahub

kubelet

maas-controller-8456fc546f-85nz4

Killing

Stopping container manager

openshift-jobset-operator

multus

jobset-controller-manager-7578997f76-wwszp

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-controller-manager-7578997f76-wwszp

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-8456fc546f from 1 to 0

opendatahub

replicaset-controller

maas-controller-8456fc546f

SuccessfulDelete

Deleted pod: maas-controller-8456fc546f-85nz4

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-jobset-operator

kubelet

jobset-controller-manager-7578997f76-wwszp

Started

Container started

openshift-jobset-operator

jobset-controller-manager-7578997f76-wwszp_9311b3ea-0a80-4ef4-970a-a6c473a400f2

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-7578997f76-wwszp_9311b3ea-0a80-4ef4-970a-a6c473a400f2 became leader

openshift-jobset-operator

kubelet

jobset-controller-manager-7578997f76-wwszp

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 3.98s (3.98s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-7578997f76-wwszp

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 12.584s (12.584s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-tt4m7

Started

Container started

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-k4vqp
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-k4vqp
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

kubelet

authorino-648d4dbc8d-djsnr

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-648d4dbc8d-djsnr

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-6f6944bcdd-k4vqp

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-6f6944bcdd-k4vqp

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-85bcdc76cd from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-648d4dbc8d from 1 to 0

kuadrant-system

replicaset-controller

authorino-648d4dbc8d

SuccessfulDelete

Deleted pod: authorino-648d4dbc8d-djsnr

kuadrant-system

replicaset-controller

authorino-648d4dbc8d

SuccessfulCreate

Created pod: authorino-648d4dbc8d-djsnr

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-648d4dbc8d from 0 to 1

kuadrant-system

replicaset-controller

authorino-85bcdc76cd

SuccessfulCreate

Created pod: authorino-85bcdc76cd-x98xg

kuadrant-system

kubelet

authorino-6f6944bcdd-k4vqp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.546s (1.546s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-85bcdc76cd-x98xg

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-85bcdc76cd-x98xg

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-648d4dbc8d-djsnr

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-k4vqp

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-6f6944bcdd-k4vqp

Started

Container started

kuadrant-system

kubelet

authorino-648d4dbc8d-djsnr

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-648d4dbc8d-djsnr

Started

Container started

kuadrant-system

kubelet

authorino-85bcdc76cd-x98xg

Started

Container started

kuadrant-system

kubelet

authorino-85bcdc76cd-x98xg

Created

Container created

kuadrant-system

kubelet

authorino-85bcdc76cd-x98xg

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.267s (1.267s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-648d4dbc8d-djsnr

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.317s (1.317s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6f6944bcdd-k4vqp

Created

Container created

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

kubelet

authorino-56c6c9ddd5-4ldrz

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-4ldrz
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Unhealthy

Startup probe failed: Get "http://10.133.0.29:15021/healthz/ready": dial tcp 10.133.0.29:15021: connect: connection refused

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-82ql9
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

ProbeError

Startup probe error: Get "http://10.133.0.29:15021/healthz/ready": dial tcp 10.133.0.29:15021: connect: connection refused body:

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

multus

llmisvc-controller-manager-6466c554d8-82ql9

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-82ql9

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-82ql9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 781ms (781ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-82ql9

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-82ql9

Started

Container started
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-cc668

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-cc668

Killing

Stopping container manager

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-rw8nw

opendatahub

multus

ai-gateway-controller-5d8dddb676-rw8nw

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-rw8nw

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-rw8nw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 2.111s (2.111s including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-rw8nw

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-rw8nw

Created

Container created

opendatahub

ai-gateway-controller-5d8dddb676-rw8nw_eabc68b1-359c-4349-b3ea-70524bf5173e

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-rw8nw_eabc68b1-359c-4349-b3ea-70524bf5173e became leader
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-86fmg

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

SuccessfulCreate

Created pod: payload-processing-7bb9489676-vzvct
(x7)

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

FailedCreate

Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-7bb9489676 from 0 to 1

openshift-ingress

kubelet

payload-processing-7bb9489676-vzvct

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-86fmg

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-7bb9489676-vzvct

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-7b98bb678c-86fmg

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-7bb9489676-vzvct

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-86fmg

Created

Container created

openshift-ingress

kubelet

payload-processing-7bb9489676-vzvct

Started

Container started

openshift-ingress

kubelet

payload-processing-7bb9489676-vzvct

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.805s (2.805s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-86fmg

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.007s (3.007s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-86fmg

Started

Container started

opendatahub

replicaset-controller

ai-gateway-controller-d69f556c7

SuccessfulCreate

Created pod: ai-gateway-controller-d69f556c7-qx7mw

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-d69f556c7 from 0 to 1

opendatahub

multus

ai-gateway-controller-d69f556c7-qx7mw

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-d69f556c7-qx7mw

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-d69f556c7-qx7mw

Created

Container created

opendatahub

kubelet

ai-gateway-controller-d69f556c7-qx7mw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 814ms (814ms including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-d69f556c7-qx7mw

Started

Container started

opendatahub

llmisvc-controller-manager-6466c554d8-82ql9_78f00233-5b47-405c-b6b5-9055b921f464

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-82ql9_78f00233-5b47-405c-b6b5-9055b921f464 became leader

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-rw8nw

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-rw8nw

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg

Created

Container created

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-zs2mg

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.202s (2.202s including waiting). Image size: 98346788 bytes.

opendatahub

ai-gateway-controller-d69f556c7-qx7mw_6021e98b-8ecb-48e4-826a-bc7da97d3ac0

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-d69f556c7-qx7mw_6021e98b-8ecb-48e4-826a-bc7da97d3ac0 became leader

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-cmqzr

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-cmqzr

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-cmqzr

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-cmqzr

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 594ms (594ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-cmqzr

Created

Container created

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-cmqzr

Started

Container started

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-ftccw

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-ftccw

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 585ms (585ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-ftccw

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-ftccw

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-ftccw

Started

Container started

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-ftccw

Created

Container created

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-pqqxp

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-pqqxp

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-pqqxp

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-pqqxp

Started

Container started

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-pqqxp

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-pqqxp

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.512s (1.512s including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq

Created

Container created

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq

Started

Container started

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8x4bcq

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 606ms (606ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-zg7ps

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

multus

echo-simulator-kserve-64b6f8f96c-zg7ps

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

kubelet

echo-simulator-kserve-64b6f8f96c-zg7ps

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

kubelet

echo-simulator-kserve-64b6f8f96c-zg7ps

Created

Container created

llm

kubelet

echo-simulator-kserve-64b6f8f96c-zg7ps

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.51s (2.51s including waiting). Image size: 78208878 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-zg7ps

Started

Container started

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 627ms (627ms including waiting). Image size: 98346788 bytes.

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c

Started

Container started

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-4kx7c

Created

Container created

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5

Started

Container started

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-x4fz5

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 589ms (589ms including waiting). Image size: 98346788 bytes.
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-6dcf757899 from 1 to 4

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-5s8q9

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-skdxg

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-skdxg

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-vtsx4

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-5s8q9

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-5s8q9

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdxg

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-vtsx4

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-vtsx4

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-5s8q9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-5s8q9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-5s8q9

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.675s (3.675s including waiting). Image size: 183555333 bytes.

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdxg

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdxg

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdxg

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.139s (4.139s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-vtsx4

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.932s (3.932s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-vtsx4

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-vtsx4

Started

Container started

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-5s8q9

ProbeError

Startup probe error: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-5s8q9

Unhealthy

Startup probe failed: Get "http://10.132.0.39:15021/healthz/ready": dial tcp 10.132.0.39:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-vtsx4

Unhealthy

Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdxg

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-vtsx4

ProbeError

Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused body:

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdxg

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-26fdx

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-6dcf757899 from 4 to 6

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-skdkh

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdkh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-skdkh

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-26fdx

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-26fdx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdkh

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdkh

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-26fdx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-26fdx

Started

Container started

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdkh

Unhealthy

Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdkh

ProbeError

Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-26fdx

Unhealthy

Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-26fdx

ProbeError

Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused body:

kuadrant-system

replicaset-controller

authorino-874888cd5

SuccessfulCreate

Created pod: authorino-874888cd5-l4bhw

kuadrant-system

kubelet

authorino-874888cd5-l4bhw

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-874888cd5-l4bhw

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-874888cd5-l4bhw

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.527s (1.527s including waiting). Image size: 204096810 bytes.
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-85bcdc76cd from 1 to 0

kuadrant-system

kubelet

authorino-874888cd5-l4bhw

Started

Container started

kuadrant-system

kubelet

authorino-874888cd5-l4bhw

Created

Container created

kuadrant-system

kubelet

authorino-85bcdc76cd-x98xg

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-85bcdc76cd

SuccessfulDelete

Deleted pod: authorino-85bcdc76cd-x98xg

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-10f834

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-10f834

openshift-ingress

replicaset-controller

e2e-worker-w4-10f834-openshift-default-65d66744fc

SuccessfulCreate

Created pod: e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-af788a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-af788a

openshift-ingress

deployment-controller

e2e-worker-w2-af788a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-af788a-openshift-default-69fcfbcd8c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-c5bec0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-c5bec0

openshift-ingress

deployment-controller

e2e-worker-w4-10f834-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-10f834-openshift-default-65d66744fc from 0 to 1

openshift-ingress

multus

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f

SuccessfulCreate

Created pod: e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

openshift-ingress

multus

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c

SuccessfulCreate

Created pod: e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

openshift-ingress

deployment-controller

e2e-worker-w3-c5bec0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-c5bec0-openshift-default-6f85d9855f from 0 to 1

openshift-ingress

multus

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-10f834

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-10f834-7bdfd44746 from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w1-90051a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-90051a-openshift-default-5d4df756c5 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-10f834: serviceaccount "payload-processing-e2e-worker-w4-10f834" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-10f834-7bdfd44746

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

openshift-ingress

replicaset-controller

e2e-worker-w1-90051a-openshift-default-5d4df756c5

SuccessfulCreate

Created pod: e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

openshift-ingress

gateway_labeler_controller

e2e-worker-w1-90051a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-90051a

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-10f834-7bdfd44746

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-10f834-7bdfd44746-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-10f834: serviceaccount "payload-processing-e2e-worker-w4-10f834" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-10f834

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-c5bec0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-36ccd3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-36ccd3

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-c5bec0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w0-36ccd3-openshift-default-f57d8465

SuccessfulCreate

Created pod: e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

openshift-ingress

deployment-controller

e2e-worker-w0-36ccd3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-36ccd3-openshift-default-f57d8465 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.957s (2.957s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

Created

Container created
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-c5bec0: serviceaccount "payload-processing-e2e-worker-w3-c5bec0" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-c5bec0: serviceaccount "payload-processing-e2e-worker-w3-c5bec0" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

openshift-ingress

multus

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

Started

Container started
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-36ccd3-8554878c58

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-36ccd3-8554878c58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-36ccd3: serviceaccount "payload-processing-e2e-worker-w0-36ccd3" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-36ccd3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-36ccd3-8554878c58 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-36ccd3-8554878c58

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-36ccd3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-36ccd3: serviceaccount "payload-processing-e2e-worker-w0-36ccd3" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.618s (2.618s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.743s (2.743s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4

Created

Container created
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-af788a: serviceaccount "payload-processing-e2e-worker-w2-af788a" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-af788a-89cdc6c6d

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-af788a-89cdc6c6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-af788a: serviceaccount "payload-processing-e2e-worker-w2-af788a" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-af788a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-af788a-89cdc6c6d

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-af788a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-af788a-89cdc6c6d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-90051a-8588c4cd7b

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-90051a-8588c4cd7b

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w1-90051a-8588c4cd7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-90051a: serviceaccount "payload-processing-e2e-worker-w1-90051a" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

Unhealthy

Readiness probe failed: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

ProbeError

Readiness probe error: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-90051a: serviceaccount "payload-processing-e2e-worker-w1-90051a" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w1-90051a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w1-90051a-8588c4cd7b from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w1-90051a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w1-90051a-cc8656f5 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

ProbeError

Startup probe error: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

Unhealthy

Startup probe failed: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-10f834-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-10f834-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

Unhealthy

Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-c5bec0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-c5bec0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-10f834-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-10f834-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-af788a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-af788a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

ProbeError

Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

ProbeError

Startup probe error: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

Unhealthy

Startup probe failed: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

ProbeError

Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-36ccd3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-36ccd3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

Unhealthy

Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-c5bec0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-c5bec0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90051a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90051a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-36ccd3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-36ccd3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-af788a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-af788a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90051a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90051a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-worker-w4-10f834-openshift-default-65d66744fc

SuccessfulCreate

Created pod: e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

openshift-ingress

multus

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

ProbeError

Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

Unhealthy

Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90051a-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w1-90051a-openshift-default-5d4df756c5

SuccessfulCreate

Created pod: e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

Started

Container started

openshift-ingress

multus

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w1-90051a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-90051a-openshift-default-5d4df756c5 from 1 to 2

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-6dcf757899 from 6 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-vtsx4

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-skdxg

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-vtsx4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdxg

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

Unhealthy

Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

ProbeError

Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused body:

opendatahub

kubelet

test-curl-7379-d99e5f

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7379-d99e5f

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7388-12987b

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7388-12987b

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7388-12987b

Started

Container started

opendatahub

kubelet

test-curl-7379-d99e5f

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.658s (2.658s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7379-d99e5f

Created

Container created

opendatahub

kubelet

test-curl-7379-d99e5f

Started

Container started

opendatahub

kubelet

test-curl-7388-12987b

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.442s (2.442s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7388-12987b

Created

Container created

opendatahub

kubelet

test-curl-7388-12987b

Killing

Stopping container test-curl-7388-12987b

opendatahub

kubelet

test-curl-7379-d99e5f

Killing

Stopping container test-curl-7379-d99e5f

openshift-ingress

replicaset-controller

e2e-extmod-a-141891-openshift-default-ddc68df65

SuccessfulCreate

Created pod: e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

openshift-ingress

deployment-controller

e2e-extmod-a-141891-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-141891-openshift-default-ddc68df65 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-141891

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-141891

openshift-ingress

replicaset-controller

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c

SuccessfulCreate

Created pod: e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

openshift-ingress

deployment-controller

e2e-extmod-a-9dda20-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-9dda20-openshift-default-84f54f8c7c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-9dda20

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-9dda20

openshift-ingress

multus

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

multus

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-9dda20

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-9dda20-7fdd45556 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-9dda20

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-9dda20-7fdd45556

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-9dda20: serviceaccount "payload-processing-e2e-extmod-a-9dda20" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-9dda20-7fdd45556

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-9dda20-7fdd45556-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-9dda20: serviceaccount "payload-processing-e2e-extmod-a-9dda20" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q

Created

Container created

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-141891

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-141891-77cfdd8587 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-141891

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-141891-699b689f5b from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-141891: serviceaccount "payload-processing-e2e-extmod-a-141891" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-141891-699b689f5b

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-141891-699b689f5b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-141891: serviceaccount "payload-processing-e2e-extmod-a-141891" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-141891-699b689f5b

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9dda20-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9dda20-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9dda20-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9dda20-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

ProbeError

Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

Unhealthy

Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-6dcf757899 from 4 to 3

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-26fdx

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-26fdx

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-d1e2ab

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-d1e2ab

openshift-ingress

deployment-controller

e2e-extmod-b-d1e2ab-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-d1e2ab-openshift-default-7d44fd885 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-13190c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-13190c

openshift-ingress

deployment-controller

e2e-extmod-a-13190c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-13190c-openshift-default-5b8c6555c from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-13190c-openshift-default-5b8c6555c

SuccessfulCreate

Created pod: e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

openshift-ingress

replicaset-controller

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885

SuccessfulCreate

Created pod: e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

openshift-ingress

multus

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

multus

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-c5bec0-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f

SuccessfulCreate

Created pod: e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

openshift-ingress

deployment-controller

e2e-worker-w3-c5bec0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-c5bec0-openshift-default-6f85d9855f from 1 to 2

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

Started

Container started

openshift-ingress

multus

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-141891-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-141891-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-491fe7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-491fe7

openshift-ingress

replicaset-controller

e2e-extmod-b-491fe7-openshift-default-7774566777

SuccessfulCreate

Created pod: e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

openshift-ingress

deployment-controller

e2e-extmod-b-491fe7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-491fe7-openshift-default-7774566777 from 0 to 1

openshift-ingress

multus

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-13190c-8589c7b8bb

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-13190c: serviceaccount "payload-processing-e2e-extmod-a-13190c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-13190c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-13190c-97548c4ff from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-13190c-8589c7b8bb

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-13190c-8589c7b8bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-13190c: serviceaccount "payload-processing-e2e-extmod-a-13190c" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-13190c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-13190c-8589c7b8bb from 0 to 1

openshift-ingress

multus

payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-d1e2ab

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-d1e2ab

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-6dcf757899 from 3 to 2
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-d1e2ab: serviceaccount "payload-processing-e2e-extmod-b-d1e2ab" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-skdkh

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-skdkh
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-d1e2ab: serviceaccount "payload-processing-e2e-extmod-b-d1e2ab" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-141891-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-141891-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-491fe7-f55cd5bb

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-491fe7-f55cd5bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-491fe7: serviceaccount "payload-processing-e2e-extmod-b-491fe7" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-491fe7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-491fe7-f55cd5bb from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-491fe7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-491fe7-85b4586559 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-491fe7: serviceaccount "payload-processing-e2e-extmod-b-491fe7" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-491fe7-f55cd5bb

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-36ccd3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-36ccd3-openshift-default-f57d8465 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-36ccd3-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-36ccd3-openshift-default-f57d8465

SuccessfulCreate

Created pod: e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

openshift-ingress

multus

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-491fe7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-491fe7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

Unhealthy

Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

ProbeError

Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused body:

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6ffc4dfcb5 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-6ffc4dfcb5

SuccessfulCreate

Created pod: payload-processing-external-model-6ffc4dfcb5-7l6vp
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-6ffc4dfcb5

FailedCreate

Error creating: pods "payload-processing-external-model-6ffc4dfcb5-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

kubelet

payload-processing-external-model-6ffc4dfcb5-7l6vp

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-6ffc4dfcb5-7l6vp

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-6ffc4dfcb5-7l6vp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-6ffc4dfcb5-7l6vp

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c

SuccessfulCreate

Created pod: e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-86b659c8d5 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-86b659c8d5

SuccessfulCreate

Created pod: payload-processing-external-model-86b659c8d5-hjmv6

models-as-a-service

multus

payload-processing-external-model-86b659c8d5-hjmv6

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-86b659c8d5-hjmv6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-86b659c8d5-hjmv6

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-86b659c8d5-hjmv6

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused body:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-6dcf757899 from 2 to 4
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

Created

Container created

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-6s4br
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-2nvtc
(x2)

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-2nvtc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-6s4br

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-6s4br

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-6s4br

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-6s4br

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-2nvtc

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-2nvtc

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-2nvtc

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-6ffc4dfcb5-7l6vp

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-6ffc4dfcb5 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-6ffc4dfcb5

SuccessfulDelete

Deleted pod: payload-processing-external-model-6ffc4dfcb5-7l6vp
(x6)

openshift-ingress

kubelet

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

Unhealthy

Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

ProbeError

Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-2nvtc

Unhealthy

Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-6s4br

ProbeError

Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-6s4br

Unhealthy

Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-2nvtc

ProbeError

Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-6dcf757899-zsxt6

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-6dcf757899 from 4 to 7

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-hj5bm

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-zsxt6

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-zsxt6

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hj5bm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-zsxt6

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hj5bm

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hj5bm

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-rmgs4

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-rmgs4

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-rmgs4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-rmgs4

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-zsxt6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w4-10f834-openshift-default-65d66744fc

SuccessfulDelete

Deleted pod: e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-cxnbf

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-10f834-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w4-10f834-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-10f834-openshift-default-65d66744fc from 2 to 1
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-13190c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-13190c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d1e2ab-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d1e2ab-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-zsxt6

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:

openshift-ingress

multus

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-zsxt6

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

Created

Container created

openshift-ingress

replicaset-controller

e2e-extmod-b-491fe7-openshift-default-7774566777

SuccessfulCreate

Created pod: e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

openshift-ingress

replicaset-controller

e2e-extmod-b-491fe7-openshift-default-7774566777

SuccessfulCreate

Created pod: e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

openshift-ingress

replicaset-controller

e2e-extmod-b-491fe7-openshift-default-7774566777

SuccessfulCreate

Created pod: e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hj5bm

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-491fe7-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-extmod-b-491fe7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-491fe7-openshift-default-7774566777 from 1 to 4
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hj5bm

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-rmgs4

ProbeError

Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-rmgs4

Unhealthy

Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused

openshift-ingress

multus

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

ProbeError

Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

Unhealthy

Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

Unhealthy

Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

ProbeError

Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

Unhealthy

Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

ProbeError

Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d1e2ab-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-13190c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d1e2ab-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-13190c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-5d459a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-5d459a

openshift-ingress

deployment-controller

e2e-extmod-b-5d459a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf

SuccessfulCreate

Created pod: e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

openshift-ingress

multus

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-859b946898 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-859b946898

SuccessfulCreate

Created pod: payload-processing-external-model-859b946898-skctn

models-as-a-service

multus

payload-processing-external-model-859b946898-skctn

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-859b946898-skctn

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-859b946898-skctn

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-859b946898-skctn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-5d459a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-5d459a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-5d459a-6df5c88b7d from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-5d459a: serviceaccount "payload-processing-e2e-extmod-b-5d459a" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-5d459a: serviceaccount "payload-processing-e2e-extmod-b-5d459a" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-491fe7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-491fe7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-extmod-b-491fe7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-491fe7-openshift-default-7774566777 from 4 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-491fe7-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-extmod-b-491fe7-openshift-default-7774566777

SuccessfulCreate

Created pod: e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-36ccd3-85c988f879-znrw4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-36ccd3-8554878c58-n9m9t

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

Unhealthy

Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

ProbeError

Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused body:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-6dcf757899 from 7 to 10

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-xgbfh

AddedInterface

Add eth0 [10.133.0.71/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-42zq8

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-swnrt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-swnrt

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-42zq8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-42zq8

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-42zq8

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-swnrt

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-xgbfh

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-xgbfh

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-xgbfh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-swnrt

Created

Container created
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-xgbfh

Unhealthy

Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-42zq8

ProbeError

Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-xgbfh

ProbeError

Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-42zq8

Unhealthy

Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-swnrt

ProbeError

Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-swnrt

Unhealthy

Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-worker-w0-36ccd3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-36ccd3-openshift-default-f57d8465 from 2 to 4

openshift-ingress

replicaset-controller

e2e-worker-w0-36ccd3-openshift-default-f57d8465

SuccessfulCreate

Created pod: e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

openshift-ingress

replicaset-controller

e2e-worker-w0-36ccd3-openshift-default-f57d8465

SuccessfulCreate

Created pod: e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-36ccd3-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

Created

Container created

openshift-ingress

multus

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

Started

Container started
(x7)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w1-90051a-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w1-90051a-openshift-default-5d4df756c5 from 2 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-90051a-openshift-default-5d4df756c5

SuccessfulDelete

Deleted pod: e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90051a-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-zdjg6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-5tjk9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-t8xx6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-gj4fp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-36ccd3-openshift-default-f57d8465-l4fst

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-141891-77cfdd8587-hbt8m

Killing

Stopping container payload-pre-processing
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-5d459a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-5d459a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-141891-699b689f5b-nmk6j

Killing

Stopping container payload-processing
(x9)

models-as-a-service

kubelet

payload-processing-external-model-86b659c8d5-hjmv6

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-bd5d7d46b00c" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9dda20-7fdd45556-fb76q

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9dda20-d969c96ff-2v6ml

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-141891-openshift-default-ddc68df65-xrds4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-9dda20-openshift-default-84f54f8c7c-rpzmw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d1e2ab-855dc6c94c-fdcc5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d1e2ab-5bd67c8c7b-ggp94

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f

SuccessfulDelete

Deleted pod: e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-xrc2g

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w3-c5bec0-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-c5bec0-openshift-default-6f85d9855f from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-c5bec0-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-491fe7-f55cd5bb-n5vmd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-491fe7-85b4586559-9wjdr

Killing

Stopping container payload-pre-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-cd8696ff5

SuccessfulCreate

Created pod: payload-processing-external-model-cd8696ff5-4976v
(x23)

models-as-a-service

kubelet

payload-processing-external-model-859b946898-skctn

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

replicaset-controller

payload-processing-external-model-859b946898

SuccessfulDelete

Deleted pod: payload-processing-external-model-859b946898-skctn

models-as-a-service

kubelet

payload-processing-external-model-859b946898-skctn

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-859b946898 from 1 to 0

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-cd8696ff5 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-cd8696ff5-4976v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-cd8696ff5-4976v

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-cd8696ff5-4976v

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-cd8696ff5-4976v

Created

Container created

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-86b659c8d5 from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-86b659c8d5-hjmv6

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-86b659c8d5

SuccessfulDelete

Deleted pod: payload-processing-external-model-86b659c8d5-hjmv6

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-ndxmn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-6957l

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-tjlv9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-bd7nm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-491fe7-openshift-default-7774566777-q7s2b

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-d1e2ab-openshift-default-7d44fd885-64cnp

Killing

Stopping container istio-proxy

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7cf9557897 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-7cf9557897

SuccessfulCreate

Created pod: payload-processing-external-model-7cf9557897-mngcx

models-as-a-service

kubelet

payload-processing-external-model-7cf9557897-mngcx

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-7cf9557897-mngcx

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-7cf9557897-mngcx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-7cf9557897-mngcx

AddedInterface

Add eth0 [10.132.0.90/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-10f834-7bdfd44746-4h4wq

Killing

Stopping container payload-processing

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-af788a-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c

SuccessfulDelete

Deleted pod: e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

openshift-ingress

deployment-controller

e2e-worker-w2-af788a-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-af788a-openshift-default-69fcfbcd8c from 2 to 1

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-9s9w8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-10f834-6d5d8b948c-fclqf

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90051a-8588c4cd7b-thnzq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

deployment-controller

e2e-worker-w4-10f834-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-10f834-openshift-default-65d66744fc from 1 to 2
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90051a-cc8656f5-6n6qz

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-worker-w4-10f834-openshift-default-65d66744fc

SuccessfulCreate

Created pod: e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-10f834-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

AddedInterface

Add eth0 [10.132.0.94/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

Started

Container started
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-af788a-89cdc6c6d-7wjwj

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-af788a-5fd4f9b6c-rx9n5

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-af788a-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c

SuccessfulCreate

Created pod: e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw
(x2)

openshift-ingress

deployment-controller

e2e-worker-w2-af788a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-af788a-openshift-default-69fcfbcd8c from 1 to 2

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

Unhealthy

Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

ProbeError

Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

ProbeError

Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

Unhealthy

Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w1-90051a-openshift-default-5d4df756c5-pnzc8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-nd22w

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w2-af788a-openshift-default-dockercfg-826jt); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w4-10f834-openshift-default-65d66744fc-b2vpp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-rrnsw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-af788a-openshift-default-69fcfbcd8c-qdb29

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x10)

models-as-a-service

kubelet

payload-processing-external-model-cd8696ff5-4976v

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-f2c682677175" not found
(x33)

models-as-a-service

kubelet

payload-processing-external-model-7cf9557897-mngcx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-7cf9557897-mngcx

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-cd8696ff5-4976v

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

ProbeError

Readiness probe error: Get "http://10.133.0.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Unhealthy

Readiness probe failed: Get "http://10.133.0.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-13190c-97548c4ff-dm59v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-13190c-8589c7b8bb-pkxrr

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-mpwlf

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-xgbfh

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-zsxt6

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-zsxt6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-xgbfh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-42zq8

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-6dcf757899-42zq8

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mpwlf

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

Killing

Stopping container istio-proxy
(x9)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-6dcf757899

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-6dcf757899-hx8xk
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-6dcf757899 from 6 to 10

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mmdl5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-hx8xk

AddedInterface

Add eth0 [10.132.0.99/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hx8xk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-w6hx4

AddedInterface

Add eth0 [10.133.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hx8xk

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-mmdl5

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mmdl5

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hx8xk

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-blsp9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mmdl5

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-6dcf757899-blsp9

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-blsp9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-w6hx4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-w6hx4

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-w6hx4

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-blsp9

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-13190c-openshift-default-5b8c6555c-7l72q

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-5d459a-6df5c88b7d-229t9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

Killing

Stopping container payload-pre-processing
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mmdl5

Unhealthy

Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-w6hx4

Unhealthy

Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-blsp9

Unhealthy

Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-mmdl5

ProbeError

Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-w6hx4

ProbeError

Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-blsp9

ProbeError

Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hx8xk

ProbeError

Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-6dcf757899-hx8xk

Unhealthy

Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-5d459a-66bdf76c4c-z6mzp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-5d459a-openshift-default-6c9dd9f4bf-kdb9p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-c5bec0-7b4f5976bd-q8ttw

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-c5bec0-ddbf5d8ff-mr6wk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

e2e-worker-w3-c5bec0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-c5bec0-openshift-default-6f85d9855f from 1 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-c5bec0-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f

SuccessfulCreate

Created pod: e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

openshift-ingress

replicaset-controller

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f

SuccessfulCreate

Created pod: e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

openshift-ingress

replicaset-controller

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f

SuccessfulCreate

Created pod: e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

Started

Container started

openshift-ingress

multus

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

AddedInterface

Add eth0 [10.133.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

AddedInterface

Add eth0 [10.132.0.102/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

Started

Container started

openshift-ingress

multus

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

AddedInterface

Add eth0 [10.132.0.103/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

Unhealthy

Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

ProbeError

Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

Unhealthy

Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

Unhealthy

Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

ProbeError

Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

ProbeError

Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-7ldgf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-j2rjt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-ltg2f

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-c5bec0-openshift-default-6f85d9855f-z2kzh

Killing

Stopping container istio-proxy