Time Namespace Component RelatedObject Reason Message

opendatahub

kserve-module-controller-manager-7566cf5fc8-zzdxv

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-r7fn5 to ip-10-0-130-91.us-east-2.compute.internal

ai-tenants

e2e-extmod-a-5c1e58

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-5c1e58: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-6dd194

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-6dd194: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-023307

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-023307: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-812032

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-812032: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-a558a6

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-a558a6: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-be2a74

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-be2a74: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-df3be8

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-df3be8: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w1-29d3ee

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w1-29d3ee: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-016e6c

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-016e6c: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-60ed4a

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-60ed4a: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-1f6ce4

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-1f6ce4: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-qpmxd

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-qpmxd to ip-10-0-142-197.us-east-2.compute.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-7ppvl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6 to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh to ip-10-0-131-6.us-east-2.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-sqp55

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-sqp55 to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5 to ip-10-0-131-6.us-east-2.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2 to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc to ip-10-0-142-197.us-east-2.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-q9whl

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-q9whl to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-7bb9489676-wzjpf

Scheduled

Successfully assigned openshift-ingress/payload-processing-7bb9489676-wzjpf to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8 to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5 to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg to ip-10-0-142-197.us-east-2.compute.internal

kuadrant-system

authorino-56c6c9ddd5-llg5c

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-llg5c to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-576b8db9c752pzq

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-576b8db9c752pzq to ip-10-0-130-91.us-east-2.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-q9dnj

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-q9dnj to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5 to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn to ip-10-0-142-197.us-east-2.compute.internal

kuadrant-system

authorino-57f7f8659f-9kxk2

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-9kxk2 to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

router-default-6d87c68c59-n7zhf

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-6d87c68c59-n7zhf

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-6d87c68c59-n7zhf

Scheduled

Successfully assigned openshift-ingress/router-default-6d87c68c59-n7zhf to ip-10-0-130-91.us-east-2.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j to ip-10-0-142-197.us-east-2.compute.internal

openshift-jobset-operator

jobset-controller-manager-6d445bbc67-m4gfl

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-6d445bbc67-m4gfl to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8 to ip-10-0-131-6.us-east-2.compute.internal

ai-tenants

e2e-extmod-a-2a8c48

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-2a8c48: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-xgjgh to ip-10-0-142-197.us-east-2.compute.internal

kuadrant-system

authorino-5ffbd6df97-qw2pm

Scheduled

Successfully assigned kuadrant-system/authorino-5ffbd6df97-qw2pm to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-rxf9k to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859 to ip-10-0-130-91.us-east-2.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-226jt

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-226jt to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-mvvkh to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-g5rgr to ip-10-0-142-197.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-646f5f8dd4-s2qzd

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd to ip-10-0-131-6.us-east-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-klvcl to ip-10-0-130-91.us-east-2.compute.internal

kuadrant-system

authorino-6889cbd8c4-4m66s

Scheduled

Successfully assigned kuadrant-system/authorino-6889cbd8c4-4m66s to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-788ls

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-788ls to ip-10-0-142-197.us-east-2.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-lrplm

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-lrplm to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-74854d8ccf-5qthm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-5qthm to ip-10-0-130-91.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-6d5d9997cc-qr9gm

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6d5d9997cc-qr9gm to ip-10-0-142-197.us-east-2.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-4rscf

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf to ip-10-0-131-6.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-7476dd8d5d-n9wd4

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7476dd8d5d-n9wd4 to ip-10-0-142-197.us-east-2.compute.internal

kuadrant-system

authorino-6dd76875b5-cdbfb

Scheduled

Successfully assigned kuadrant-system/authorino-6dd76875b5-cdbfb to ip-10-0-131-6.us-east-2.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-z9lc6

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-z9lc6 to ip-10-0-130-91.us-east-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-5nk76 to ip-10-0-142-197.us-east-2.compute.internal

kuadrant-system

authorino-7cc7fd6fd6-lmgkl

Scheduled

Successfully assigned kuadrant-system/authorino-7cc7fd6fd6-lmgkl to ip-10-0-142-197.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-8474b49dbc-dvj9v

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-8474b49dbc-dvj9v to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

kube-auth-proxy-589fcc7ff4-dcwx7

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7 to ip-10-0-131-6.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-8474b49dbc-pxhg8

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-8474b49dbc-pxhg8 to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-d7xzv

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-8j245

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8j245 to ip-10-0-130-91.us-east-2.compute.internal

models-as-a-service

payload-processing-external-model-9699b4dc9-g954t

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-9699b4dc9-g954t to ip-10-0-131-6.us-east-2.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-fmj2k

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-fmj2k to ip-10-0-130-91.us-east-2.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-v2t9p

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-v2t9p to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-7kvxn

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g to ip-10-0-130-91.us-east-2.compute.internal

opendatahub

ai-gateway-controller-5d8dddb676-tzffp

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-tzffp to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr to ip-10-0-130-91.us-east-2.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp to ip-10-0-130-91.us-east-2.compute.internal

opendatahub

ai-gateway-controller-84bbcb8bf9-hjnv7

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-84bbcb8bf9-hjnv7 to ip-10-0-142-197.us-east-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9 to ip-10-0-142-197.us-east-2.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-zgbmm

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2 to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc to ip-10-0-142-197.us-east-2.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-jtwwh

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-jtwwh to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9 to ip-10-0-130-91.us-east-2.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-4c2q5

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-4c2q5 to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f to ip-10-0-131-6.us-east-2.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-clnnz

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-clnnz to ip-10-0-131-6.us-east-2.compute.internal

ai-tenants

e2e-extmod-a-3da894

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-3da894: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

kube-auth-proxy-589fcc7ff4-h6ggf

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-589fcc7ff4-h6ggf to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz to ip-10-0-131-6.us-east-2.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-xwnp7

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-xwnp7 to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b to ip-10-0-131-6.us-east-2.compute.internal

opendatahub

test-curl-11453-a07e6d

Scheduled

Successfully assigned opendatahub/test-curl-11453-a07e6d to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5 to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9 to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz to ip-10-0-130-91.us-east-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-rntjw

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rntjw to ip-10-0-130-91.us-east-2.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-pw7r7

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-pw7r7 to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x to ip-10-0-142-197.us-east-2.compute.internal

opendatahub

test-curl-11453-5385ed

Scheduled

Successfully assigned opendatahub/test-curl-11453-5385ed to ip-10-0-130-91.us-east-2.compute.internal

opendatahub

odh-model-controller-769875db56-mcg7w

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-mcg7w to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp to ip-10-0-142-197.us-east-2.compute.internal

opendatahub

model-serving-api-75bd89fdfc-msppb

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-msppb to ip-10-0-142-197.us-east-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-sqpgg

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sqpgg to ip-10-0-130-91.us-east-2.compute.internal

opendatahub

maas-controller-779b45f986-ng2mv

Scheduled

Successfully assigned opendatahub/maas-controller-779b45f986-ng2mv to ip-10-0-131-6.us-east-2.compute.internal

opendatahub

maas-controller-5f5c75bcb9-lnh6s

Scheduled

Successfully assigned opendatahub/maas-controller-5f5c75bcb9-lnh6s to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl to ip-10-0-130-91.us-east-2.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-94g7x

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-94g7x to ip-10-0-131-6.us-east-2.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-xccjr

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-xccjr to ip-10-0-130-91.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc to ip-10-0-131-6.us-east-2.compute.internal

openshift-ingress

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz to ip-10-0-142-197.us-east-2.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-6d87c68c59 from 0 to 1

openshift-ingress

replicaset-controller

router-default-6d87c68c59

SuccessfulCreate

Created pod: router-default-6d87c68c59-n7zhf

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-6d87c68c59-n7zhf

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-6d87c68c59-n7zhf

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x4)

openshift-ingress

kubelet

router-default-6d87c68c59-n7zhf

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-6d87c68c59-n7zhf

AddedInterface

Add eth0 [10.132.0.9/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-6d87c68c59-n7zhf

Started

Container started

openshift-ingress

kubelet

router-default-6d87c68c59-n7zhf

Created

Container created

openshift-ingress

kubelet

router-default-6d87c68c59-n7zhf

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-7kvxn

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-7kvxn

AddedInterface

Add eth0 [10.133.0.19/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-7kvxn

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-7kvxn

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-7kvxn

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.448s (3.448s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-7kvxn

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-7kvxn

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-7kvxn

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-fc457bc99

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-fc457bc99 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-xgjgh

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.418s (3.418s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859

Started

Container started

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-fc457bc99-bj859

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 6.649s (6.649s including waiting). Image size: 183555333 bytes.

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-lrplm
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-lrplm

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-lrplm

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-lrplm

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-lrplm

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 3.035s (3.035s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-lrplm_4bd00467-8b4a-4a07-9cf6-773c63d59b3c became leader

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-lrplm

Created

Container created

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-8f94f88d4-5nk76

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-8f94f88d4

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-8f94f88d4-5nk76

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-8f94f88d4

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-8f94f88d4-2xfwc

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-8f94f88d4 from 0 to 3

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-8f94f88d4

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-8f94f88d4-klvcl

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

NeedsReinstall

calculated deployment install is bad

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-8f94f88d4-klvcl

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

operator-lifecycle-manager

install-4pnhg

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.803s (2.803s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.835s (2.835s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Started

Container started
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.895s (2.895s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 428ms (428ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 549ms (549ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 515ms (515ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-2xfwc

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-klvcl

Started

Container started

opendatahub-operator-system

opendatahub-operator-controller-manager-8f94f88d4-klvcl_30f74a6e-5b20-43ab-afed-d68261358d81

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-8f94f88d4-klvcl_30f74a6e-5b20-43ab-afed-d68261358d81 became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8f94f88d4-5nk76

Created

Container created

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-q9whl

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-q9whl

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-q9whl

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-fmj2k
(x8)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

kuadrant-system

multus

authorino-operator-7b7bb4bc77-fmj2k

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fmj2k

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-sqp55

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-sqp55

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fmj2k

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 2.821s (2.821s including waiting). Image size: 167738465 bytes.

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fmj2k

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fmj2k

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-q9whl

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.154s (3.154s including waiting). Image size: 309650901 bytes.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-sqp55

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-sqp55

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 1.746s (1.746s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-sqp55

Created

Container created

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-q9whl

Started

Container started

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-jtwwh

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

authorino-operator-7b7bb4bc77-fmj2k_5a8934a8-6711-4005-bcc1-59bca0260c97

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-fmj2k_5a8934a8-6711-4005-bcc1-59bca0260c97 became leader

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-q9whl

Created

Container created

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-jtwwh

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jtwwh

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-sqp55

Started

Container started

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

dns-operator-controller-manager-8557cbdb84-jtwwh_4029d47a-b140-43d8-bde6-e35541b38c19

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-jtwwh_4029d47a-b140-43d8-bde6-e35541b38c19 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jtwwh

Started

Container started

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jtwwh

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jtwwh

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 2.386s (2.386s including waiting). Image size: 190804149 bytes.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-4c2q5

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-4c2q5

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.
(x9)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-4c2q5

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-qpmxd

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-4c2q5

Started

Container started

cert-manager

multus

cert-manager-597b95cb84-qpmxd

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-qpmxd

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

cert-manager

kubelet

cert-manager-597b95cb84-qpmxd

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-4c2q5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 1.947s (1.947s including waiting). Image size: 177967631 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-4c2q5

Created

Container created

kuadrant-system

limitador-operator-controller-manager-5778bb4449-4c2q5_ef1080ba-36ad-4f98-8353-66c96a723e71

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-4c2q5_ef1080ba-36ad-4f98-8353-66c96a723e71 became leader

cert-manager

kubelet

cert-manager-597b95cb84-qpmxd

Started

Container started

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-rntjw

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rntjw

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-rntjw

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rntjw

Started

Container started

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-rntjw_5dc5b305-b46d-401e-8554-601b3076199f

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-rntjw_5dc5b305-b46d-401e-8554-601b3076199f became leader

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rntjw

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rntjw

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 4.307s (4.307s including waiting). Image size: 424244516 bytes.

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-576b8db9c7

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-576b8db9c752pzq

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-576b8db9c752pzq

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-576b8db9c752pzq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-h6ggf

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-576b8db9c7 from 0 to 1

openshift-ingress

multus

kube-auth-proxy-589fcc7ff4-h6ggf

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-dcwx7

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

replicaset-controller

kube-auth-proxy-589fcc7ff4

SuccessfulCreate

Created pod: kube-auth-proxy-589fcc7ff4-dcwx7

openshift-ingress

replicaset-controller

kube-auth-proxy-589fcc7ff4

SuccessfulCreate

Created pod: kube-auth-proxy-589fcc7ff4-h6ggf

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-589fcc7ff4 from 0 to 2

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-576b8db9c752pzq

Created

Container created

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-ingress

multus

kube-auth-proxy-589fcc7ff4-dcwx7

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-dcwx7

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-576b8db9c752pzq

Started

Container started

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-h6ggf

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.539s (2.539s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-h6ggf

Created

Container created

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-zzdxv

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-h6ggf

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-dcwx7

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-dcwx7

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-589fcc7ff4-dcwx7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.336s (2.336s including waiting). Image size: 188904677 bytes.

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-zzdxv

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-zzdxv

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-zzdxv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 1.379s (1.379s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-zzdxv

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-zzdxv

Started

Container started
(x5)

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreateFailed

Failed to create Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator: Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": tls: failed to verify certificate: x509: certificate signed by unknown authority

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kserve-module-controller-manager-7566cf5fc8-zzdxv_0ab20ead-6c9a-4df5-b4fa-446adfdd31ab

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-zzdxv_0ab20ead-6c9a-4df5-b4fa-446adfdd31ab became leader
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

kubelet

model-serving-api-75bd89fdfc-msppb

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-mcg7w

opendatahub

multus

model-serving-api-75bd89fdfc-msppb

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

opendatahub

multus

kserve-controller-manager-776cc66d47-clnnz

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-d7xzv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-d7xzv

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-xccjr

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-xccjr

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-xccjr

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-d7xzv

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

opendatahub

kubelet

odh-model-controller-769875db56-mcg7w

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

multus

odh-model-controller-769875db56-mcg7w

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-msppb

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-clnnz

opendatahub

kubelet

kserve-controller-manager-776cc66d47-clnnz

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-d7xzv

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-d7xzv

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-7kvxn

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-7kvxn

opendatahub

kubelet

kserve-controller-manager-776cc66d47-clnnz

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 1.952s (1.952s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-clnnz

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-clnnz

Started

Container started

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

opendatahub

kubelet

model-serving-api-75bd89fdfc-msppb

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-msppb

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-msppb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 1.634s (1.634s including waiting). Image size: 184925761 bytes.

opendatahub

odh-model-controller-769875db56-mcg7w_28580456-212f-46eb-8a8e-b49672f71497

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-mcg7w_28580456-212f-46eb-8a8e-b49672f71497 became leader

opendatahub

kubelet

odh-model-controller-769875db56-mcg7w

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.322s (3.322s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-mcg7w

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-mcg7w

Started

Container started

opendatahub

kserve-controller-manager-776cc66d47-clnnz_aa3e654f-b3eb-4e3a-9ad3-9fc4804acba3

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-clnnz_aa3e654f-b3eb-4e3a-9ad3-9fc4804acba3 became leader

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-xccjr

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.361s (3.361s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-xccjr

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-xccjr

Started

Container started

opendatahub

llmisvc-controller-manager-869d5dcc5d-xccjr_b2a3477d-633e-4113-bdda-d0be067a1ba9

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-xccjr_b2a3477d-633e-4113-bdda-d0be067a1ba9 became leader

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-cjv8g"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-j8xbf"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

multus

jobset-controller-manager-6d445bbc67-m4gfl

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-controller-manager-6d445bbc67-m4gfl

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-6d445bbc67 from 0 to 1

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-6d445bbc67

SuccessfulCreate

Created pod: jobset-controller-manager-6d445bbc67-m4gfl

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-jobset-operator

jobset-controller-manager-6d445bbc67-m4gfl_b0c1d3a7-6103-488b-8691-fb2854deb89c

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-6d445bbc67-m4gfl_b0c1d3a7-6103-488b-8691-fb2854deb89c became leader

openshift-jobset-operator

kubelet

jobset-controller-manager-6d445bbc67-m4gfl

Started

Container started

openshift-jobset-operator

kubelet

jobset-controller-manager-6d445bbc67-m4gfl

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-6d445bbc67-m4gfl

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 3.72s (3.72s including waiting). Image size: 182184678 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

NeedsReinstall

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rntjw

Killing

Stopping container manager

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

ComponentUnhealthy

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-sqpgg

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-sqpgg

Created

Container created

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-sqpgg

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-sqpgg

Started

Container started
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-sqpgg

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-sqpgg_906dd6d2-ee31-4c78-8d0b-f330e65203cb

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-sqpgg_906dd6d2-ee31-4c78-8d0b-f330e65203cb became leader

kuadrant-system

multus

limitador-limitador-74dfcfb894-94g7x

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-9kxk2

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-z9lc6

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-xwnp7

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-xwnp7

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-94g7x

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-xwnp7

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-94g7x

kuadrant-system

kubelet

authorino-56c6c9ddd5-llg5c

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

kubelet

authorino-57f7f8659f-9kxk2

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-z9lc6

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

multus

limitador-limitador-7c9b87d574-z9lc6

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

kuadrant-system

multus

authorino-57f7f8659f-9kxk2

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-llg5c

kuadrant-system

multus

authorino-56c6c9ddd5-llg5c

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-94g7x

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-94g7x

Started

Container started

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-94g7x

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 3.098s (3.098s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-llg5c

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-9kxk2

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-9kxk2

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-9kxk2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.65s (2.65s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-llg5c

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-llg5c

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.727s (2.727s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-xwnp7

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-xwnp7

Created

Container created

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-9kxk2

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-xwnp7

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 4.865s (4.865s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

authorino-57f7f8659f-9kxk2

Killing

Stopping container authorino

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-z9lc6

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 5.721s (5.721s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-z9lc6

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-z9lc6

Started

Container started

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-94g7x

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-94g7x

Killing

Stopping container limitador

opendatahub

multus

maas-controller-5f5c75bcb9-lnh6s

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

opendatahub

replicaset-controller

maas-controller-779b45f986

SuccessfulCreate

Created pod: maas-controller-779b45f986-ng2mv

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-5f5c75bcb9 from 0 to 1

opendatahub

replicaset-controller

maas-controller-5f5c75bcb9

SuccessfulCreate

Created pod: maas-controller-5f5c75bcb9-lnh6s

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-779b45f986 from 0 to 1

opendatahub

kubelet

maas-controller-5f5c75bcb9-lnh6s

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a57ee5e7c780d1b6e01feb873bb20edd0430d61023a212b78fc075ac9dc16c90"

opendatahub

multus

maas-controller-779b45f986-ng2mv

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-779b45f986-ng2mv

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a57ee5e7c780d1b6e01feb873bb20edd0430d61023a212b78fc075ac9dc16c90"

opendatahub

kubelet

maas-controller-779b45f986-ng2mv

Started

Container started

opendatahub

kubelet

maas-controller-779b45f986-ng2mv

Created

Container created

opendatahub

kubelet

maas-controller-5f5c75bcb9-lnh6s

Started

Container started

opendatahub

kubelet

maas-controller-5f5c75bcb9-lnh6s

Created

Container created

opendatahub

kubelet

maas-controller-779b45f986-ng2mv

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a57ee5e7c780d1b6e01feb873bb20edd0430d61023a212b78fc075ac9dc16c90" in 3.625s (3.625s including waiting). Image size: 200142869 bytes.

opendatahub

kubelet

maas-controller-5f5c75bcb9-lnh6s

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a57ee5e7c780d1b6e01feb873bb20edd0430d61023a212b78fc075ac9dc16c90" in 4.021s (4.021s including waiting). Image size: 200142869 bytes.

opendatahub

maas-controller-779b45f986-ng2mv_c9f3d526-6e3c-41c1-902e-04ed7befa689

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-779b45f986-ng2mv_c9f3d526-6e3c-41c1-902e-04ed7befa689 became leader

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-zgbmm

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zgbmm

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-zgbmm

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

opendatahub

replicaset-controller

maas-controller-5f5c75bcb9

SuccessfulDelete

Deleted pod: maas-controller-5f5c75bcb9-lnh6s

opendatahub

kubelet

maas-controller-5f5c75bcb9-lnh6s

Killing

Stopping container manager

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-5f5c75bcb9 from 1 to 0

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zgbmm

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 16.257s (16.257s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zgbmm

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zgbmm

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Started

Container started
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Created

Container created

kuadrant-system

replicaset-controller

authorino-6dd76875b5

SuccessfulDelete

Deleted pod: authorino-6dd76875b5-cdbfb

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6889cbd8c4 from 1 to 0

kuadrant-system

multus

authorino-6889cbd8c4-4m66s

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-5ffbd6df97

SuccessfulCreate

Created pod: authorino-5ffbd6df97-qw2pm

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulCreate

Created pod: authorino-6889cbd8c4-4m66s

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulDelete

Deleted pod: authorino-6889cbd8c4-4m66s

kuadrant-system

replicaset-controller

authorino-6dd76875b5

SuccessfulCreate

Created pod: authorino-6dd76875b5-cdbfb

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-5ffbd6df97 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6889cbd8c4 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6dd76875b5 from 1 to 0

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6dd76875b5 from 0 to 1

kuadrant-system

multus

authorino-5ffbd6df97-qw2pm

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-6889cbd8c4-4m66s

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-5ffbd6df97-qw2pm

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-5ffbd6df97-qw2pm

Created

Container created

kuadrant-system

kubelet

authorino-6889cbd8c4-4m66s

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.158s (1.158s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-5ffbd6df97-qw2pm

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 683ms (683ms including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-5ffbd6df97-qw2pm

Started

Container started

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-llg5c

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

kubelet

authorino-6889cbd8c4-4m66s

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-6889cbd8c4-4m66s

Created

Container created

kuadrant-system

kubelet

authorino-6889cbd8c4-4m66s

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-llg5c

Killing

Stopping container authorino

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-pw7r7

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Unhealthy

Startup probe failed: Get "http://10.134.0.22:15021/healthz/ready": dial tcp 10.134.0.22:15021: connect: connection refused

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-pw7r7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 407ms (407ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-pw7r7

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-pw7r7

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-pw7r7

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-6466c554d8-pw7r7

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

ProbeError

Startup probe error: Get "http://10.134.0.22:15021/healthz/ready": dial tcp 10.134.0.22:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-xccjr

Killing

Stopping container manager

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-xccjr

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-tzffp

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

multus

ai-gateway-controller-5d8dddb676-tzffp

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tzffp

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tzffp

Created

Container created

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tzffp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 2.422s (2.422s including waiting). Image size: 184136100 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tzffp

Started

Container started

opendatahub

ai-gateway-controller-5d8dddb676-tzffp_84de9db5-2378-4f21-b7a3-b4692f59248b

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-tzffp_84de9db5-2378-4f21-b7a3-b4692f59248b became leader

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-7ppvl

openshift-ingress

kubelet

payload-processing-7bb9489676-wzjpf

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"
(x3)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-7bb9489676 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

SuccessfulCreate

Created pod: payload-processing-7bb9489676-wzjpf
(x4)

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

FailedCreate

Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

multus

payload-processing-7bb9489676-wzjpf

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7ppvl

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-7b98bb678c-7ppvl

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-7bb9489676-wzjpf

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.459s (2.459s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-7bb9489676-wzjpf

Created

Container created

openshift-ingress

kubelet

payload-processing-7bb9489676-wzjpf

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7ppvl

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.372s (2.372s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7ppvl

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7ppvl

Started

Container started

opendatahub

multus

ai-gateway-controller-84bbcb8bf9-hjnv7

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-84bbcb8bf9-hjnv7

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

replicaset-controller

ai-gateway-controller-84bbcb8bf9

SuccessfulCreate

Created pod: ai-gateway-controller-84bbcb8bf9-hjnv7

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-84bbcb8bf9 from 0 to 1

opendatahub

kubelet

ai-gateway-controller-84bbcb8bf9-hjnv7

Created

Container created

opendatahub

kubelet

ai-gateway-controller-84bbcb8bf9-hjnv7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 386ms (386ms including waiting). Image size: 184136100 bytes.

opendatahub

llmisvc-controller-manager-6466c554d8-pw7r7_50faddd6-a27e-41ee-8f71-4295aeea7bf7

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-pw7r7_50faddd6-a27e-41ee-8f71-4295aeea7bf7 became leader

opendatahub

kubelet

ai-gateway-controller-84bbcb8bf9-hjnv7

Started

Container started

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tzffp

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-tzffp

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.305s (1.305s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc

Created

Container created

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc

Started

Container started

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-v2t9p

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-v2t9p

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-v2t9p

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-v2t9p

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.228s (1.228s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-v2t9p

Started

Container started

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-v2t9p

Created

Container created

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-4rscf

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-226jt

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 283ms (283ms including waiting). Image size: 98346788 bytes.

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-226jt

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4rscf

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4rscf

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 290ms (290ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4rscf

Created

Container created

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4rscf

Started

Container started

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-4rscf

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-226jt

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-226jt

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 301ms (301ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-226jt

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-226jt

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8ps68j

Started

Container started

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

multus

echo-simulator-kserve-64b6f8f96c-q9dnj

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-q9dnj

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

kubelet

echo-simulator-kserve-64b6f8f96c-q9dnj

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

echo-simulator-kserve-64b6f8f96c-q9dnj

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.044s (2.044s including waiting). Image size: 78208878 bytes.

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

kubelet

echo-simulator-kserve-64b6f8f96c-q9dnj

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2

Started

Container started

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2

Created

Container created

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 329ms (329ms including waiting). Image size: 98346788 bytes.

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

opendatahub

ai-gateway-controller-84bbcb8bf9-hjnv7_810d559d-5b4f-41a1-b3df-6f3108cd53ff

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-84bbcb8bf9-hjnv7_810d559d-5b4f-41a1-b3df-6f3108cd53ff became leader

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Started

Container started

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Created

Container created

llm

kubelet

echo-simulator-kserve-64b6f8f96c-q9dnj

Started

Container started

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 298ms (298ms including waiting). Image size: 98346788 bytes.
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

kuadrant-system

replicaset-controller

authorino-7cc7fd6fd6

SuccessfulCreate

Created pod: authorino-7cc7fd6fd6-lmgkl

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

kuadrant-system

multus

authorino-7cc7fd6fd6-lmgkl

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-7cc7fd6fd6-lmgkl

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-7cc7fd6fd6-lmgkl

Created

Container created

kuadrant-system

kubelet

authorino-7cc7fd6fd6-lmgkl

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 1 to 4

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-5qthm

kuadrant-system

kubelet

authorino-7cc7fd6fd6-lmgkl

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 939ms (939ms including waiting). Image size: 204096810 bytes.

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-rxf9k

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-lhm7v

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-5qthm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-5ffbd6df97 from 1 to 0

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-5qthm

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-5qthm

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-5qthm

Started

Container started

kuadrant-system

replicaset-controller

authorino-5ffbd6df97

SuccessfulDelete

Deleted pod: authorino-5ffbd6df97-qw2pm

kuadrant-system

kubelet

authorino-5ffbd6df97-qw2pm

Killing

Stopping container authorino

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.078s (3.078s including waiting). Image size: 183555333 bytes.
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

ProbeError

Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-5qthm

ProbeError

Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-5qthm

Unhealthy

Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

Unhealthy

Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-7wfxn

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 4 to 6

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-r7fn5

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-1f6ce4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-1f6ce4

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-60ed4a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-60ed4a

openshift-ingress

deployment-controller

e2e-worker-w3-60ed4a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-60ed4a-openshift-default-76f5998f8c from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-016e6c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-016e6c

openshift-ingress

deployment-controller

e2e-worker-w2-016e6c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-016e6c-openshift-default-644bc689bc from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c

SuccessfulCreate

Created pod: e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w4-1f6ce4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-df3be8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-df3be8

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-df3be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-df3be8-openshift-default-854766f5d9 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4: serviceaccount "payload-processing-e2e-worker-w4-1f6ce4" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-1f6ce4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-1f6ce4: serviceaccount "payload-processing-e2e-worker-w4-1f6ce4" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-1f6ce4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-016e6c-68fd9678fb

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-016e6c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-016e6c-68fd9678fb from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-016e6c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-016e6c: serviceaccount "payload-processing-e2e-worker-w2-016e6c" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-016e6c-68fd9678fb

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-016e6c-68fd9678fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-016e6c: serviceaccount "payload-processing-e2e-worker-w2-016e6c" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-60ed4a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-60ed4a-fd7969596 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-60ed4a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-60ed4a: serviceaccount "payload-processing-e2e-worker-w3-60ed4a" not found

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-60ed4a-fd7969596

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-60ed4a-fd7969596-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-60ed4a: serviceaccount "payload-processing-e2e-worker-w3-60ed4a" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-60ed4a-fd7969596

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-df3be8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-df3be8-5f967dd678 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-df3be8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-df3be8: serviceaccount "payload-processing-e2e-worker-w0-df3be8" not found

openshift-ingress

multus

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-df3be8-5f967dd678

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-df3be8-5f967dd678-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-df3be8: serviceaccount "payload-processing-e2e-worker-w0-df3be8" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-df3be8-5f967dd678

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

openshift-ingress

multus

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

deployment-controller

e2e-worker-w1-29d3ee-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-29d3ee-openshift-default-844cdd95d7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7

SuccessfulCreate

Created pod: e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

openshift-ingress

gateway_labeler_controller

e2e-worker-w1-29d3ee

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-29d3ee

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.87s (4.87s including waiting). Image size: 125353963 bytes.

openshift-ingress

multus

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.172s (3.172s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.292s (3.292s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.659s (4.659s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.705s (4.705s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.719s (4.719s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.74s (2.74s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.847s (2.847s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

Created

Container created
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-29d3ee: serviceaccount "payload-processing-e2e-worker-w1-29d3ee" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w1-29d3ee

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-29d3ee: serviceaccount "payload-processing-e2e-worker-w1-29d3ee" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w1-29d3ee

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w1-29d3ee-6b69556f7b from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Unhealthy

Readiness probe failed: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

ProbeError

Readiness probe error: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-1f6ce4-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-1f6ce4-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

ProbeError

Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Unhealthy

Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

ProbeError

Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

Unhealthy

Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

ProbeError

Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

Unhealthy

Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

ProbeError

Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

Unhealthy

Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-29d3ee-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-29d3ee-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

ProbeError

Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

Unhealthy

Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-1f6ce4-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-1f6ce4-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-60ed4a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-60ed4a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w2-016e6c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-016e6c-openshift-default-644bc689bc from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

Created

Container created

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

ProbeError

Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

Unhealthy

Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-29d3ee-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-29d3ee-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c

SuccessfulCreate

Created pod: e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w4-1f6ce4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-1f6ce4-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

Created

Container created

openshift-ingress

multus

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

Started

Container started

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w0-df3be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-df3be8-openshift-default-854766f5d9 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-29d3ee-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

Started

Container started

openshift-ingress

multus

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w1-29d3ee-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-29d3ee-openshift-default-844cdd95d7 from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7

SuccessfulCreate

Created pod: e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p
(x6)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

ProbeError

Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

Unhealthy

Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-60ed4a-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

Started

Container started

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w3-60ed4a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-60ed4a-openshift-default-76f5998f8c from 1 to 2
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

ProbeError

Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

Unhealthy

Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused

opendatahub

kubelet

test-curl-11453-a07e6d

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-11453-a07e6d

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-11453-a07e6d

Started

Container started

opendatahub

kubelet

test-curl-11453-a07e6d

Created

Container created

opendatahub

kubelet

test-curl-11453-a07e6d

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 967ms (967ms including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-11453-5385ed

Started

Container started

opendatahub

multus

test-curl-11453-5385ed

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-11453-5385ed

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-11453-a07e6d

Killing

Stopping container test-curl-11453-a07e6d

opendatahub

kubelet

test-curl-11453-5385ed

Created

Container created

opendatahub

kubelet

test-curl-11453-5385ed

Killing

Stopping container test-curl-11453-5385ed

openshift-ingress

deployment-controller

e2e-extmod-a-6dd194-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-6dd194-openshift-default-6f76f45889 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-6dd194

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-6dd194

openshift-ingress

replicaset-controller

e2e-extmod-a-6dd194-openshift-default-6f76f45889

SuccessfulCreate

Created pod: e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

openshift-ingress

deployment-controller

e2e-worker-w2-016e6c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-016e6c-openshift-default-644bc689bc from 2 to 3

openshift-ingress

multus

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-6dd194-5794949964

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-6dd194

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-6dd194-5794949964 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-6dd194

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-6dd194-5794949964

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-6dd194-5794949964-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-6dd194: serviceaccount "payload-processing-e2e-extmod-a-6dd194" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-6dd194: serviceaccount "payload-processing-e2e-extmod-a-6dd194" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

Unhealthy

Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

ProbeError

Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

openshift-ingress

deployment-controller

e2e-worker-w0-df3be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-df3be8-openshift-default-854766f5d9 from 2 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Created

Container created

openshift-ingress

deployment-controller

e2e-extmod-b-812032-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-812032-openshift-default-9f8549b85 from 0 to 1
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Unhealthy

Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-extmod-b-812032-openshift-default-9f8549b85

SuccessfulCreate

Created pod: e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

ProbeError

Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-812032

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-812032
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused body:

openshift-ingress

multus

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Unhealthy

Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

ProbeError

Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-812032-99cfb6886

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-812032

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-812032-99cfb6886 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-812032: serviceaccount "payload-processing-e2e-extmod-b-812032" not found
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-812032-99cfb6886

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-812032-99cfb6886-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-812032: serviceaccount "payload-processing-e2e-extmod-b-812032" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-812032

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-812032-6c4759db6c from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w2-016e6c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-016e6c-openshift-default-644bc689bc from 3 to 4

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w3-60ed4a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-60ed4a-openshift-default-76f5998f8c from 2 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-60ed4a-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

openshift-ingress

multus

payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

Started

Container started

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Unhealthy

Readiness probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

ProbeError

Readiness probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Started

Container started

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-2a8c48

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-2a8c48

openshift-ingress

deployment-controller

e2e-extmod-a-3da894-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-3da894-openshift-default-695bd64b77 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-3da894

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-3da894

openshift-ingress

replicaset-controller

e2e-extmod-a-3da894-openshift-default-695bd64b77

SuccessfulCreate

Created pod: e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

openshift-ingress

replicaset-controller

e2e-extmod-a-2a8c48-openshift-default-5d7b54765

SuccessfulCreate

Created pod: e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

openshift-ingress

deployment-controller

e2e-extmod-a-2a8c48-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-2a8c48-openshift-default-5d7b54765 from 0 to 1

openshift-ingress

multus

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

multus

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

Unhealthy

Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

ProbeError

Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

ProbeError

Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

Unhealthy

Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

ProbeError

Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Unhealthy

Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused
(x5)

models-as-a-service

replicaset-controller

payload-processing-external-model-646f5f8dd4

FailedCreate

Error creating: pods "payload-processing-external-model-646f5f8dd4-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-646f5f8dd4 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-646f5f8dd4

SuccessfulCreate

Created pod: payload-processing-external-model-646f5f8dd4-s2qzd

models-as-a-service

kubelet

payload-processing-external-model-646f5f8dd4-s2qzd

Started

Container started
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-2a8c48: serviceaccount "payload-processing-e2e-extmod-a-2a8c48" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-3da894

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-3da894-966b64c9d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-2a8c48

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-2a8c48: serviceaccount "payload-processing-e2e-extmod-a-2a8c48" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-2a8c48

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-646f5f8dd4-s2qzd

Created

Container created

models-as-a-service

multus

payload-processing-external-model-646f5f8dd4-s2qzd

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-646f5f8dd4-s2qzd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf

AddedInterface

Add eth0 [10.133.0.64/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-3da894: serviceaccount "payload-processing-e2e-extmod-a-3da894" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt

Created

Container created
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-3da894-966b64c9d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-3da894-966b64c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-3da894: serviceaccount "payload-processing-e2e-extmod-a-3da894" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-3da894-966b64c9d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-3da894

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w2-016e6c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-016e6c-openshift-default-644bc689bc from 4 to 8

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

openshift-ingress

deployment-controller

e2e-worker-w0-df3be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-df3be8-openshift-default-854766f5d9 from 4 to 5

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

Unhealthy

Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

ProbeError

Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

Unhealthy

Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

Unhealthy

Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

ProbeError

Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

ProbeError

Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6dd194-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6dd194-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-60ed4a-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w3-60ed4a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-60ed4a-openshift-default-76f5998f8c from 4 to 5

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

Started

Container started

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-788ls

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-788ls

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

Started

Container started

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-74854d8ccf from 6 to 10

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-g5rgr

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-mvvkh

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-74854d8ccf-bx8vz
(x2)

openshift-ingress

kubelet

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Unhealthy

Readiness probe failed: Get "https://10.134.0.38:8000/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-023307

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-023307

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-be2a74

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-be2a74

openshift-ingress

replicaset-controller

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7

SuccessfulCreate

Created pod: e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

openshift-ingress

deployment-controller

e2e-extmod-b-be2a74-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-be2a74-openshift-default-5f4f5978f7 from 0 to 1

openshift-ingress

deployment-controller

e2e-extmod-b-023307-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-023307-openshift-default-645c679bbf from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-023307-openshift-default-645c679bbf

SuccessfulCreate

Created pod: e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

openshift-ingress

multus

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-812032-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-812032-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x10)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused
(x10)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-8j245

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

ProbeError

Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused body:

openshift-ingress

multus

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

Unhealthy

Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-8j245

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8j245

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"
(x8)

openshift-ingress

kubelet

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

ProbeError

Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

Unhealthy

Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused
(x10)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

ProbeError

Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused body:
(x10)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

Unhealthy

Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused
(x12)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:
(x12)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x12)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

ProbeError

Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

ProbeError

Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x12)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

Unhealthy

Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6dd194-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6dd194-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-be2a74

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-be2a74

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-be2a74: serviceaccount "payload-processing-e2e-extmod-b-be2a74" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-bx8vz

Unhealthy

Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-be2a74: serviceaccount "payload-processing-e2e-extmod-b-be2a74" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

ProbeError

Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

Unhealthy

Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-023307-8c4d6b74d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-023307-8c4d6b74d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-023307-8c4d6b74d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-023307: serviceaccount "payload-processing-e2e-extmod-b-023307" not found

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8j245

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.023s (4.023s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8j245

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

ProbeError

Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8j245

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-023307

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-023307

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-023307-8c4d6b74d from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-023307: serviceaccount "payload-processing-e2e-extmod-b-023307" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

ProbeError

Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

Unhealthy

Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

Unhealthy

Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

ProbeError

Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf

SuccessfulCreate

Created pod: e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

openshift-ingress

deployment-controller

e2e-extmod-a-5c1e58-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-5c1e58

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-5c1e58

openshift-ingress

multus

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-3da894-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-3da894-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-2a8c48-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-2a8c48-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-5c1e58

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-5c1e58: serviceaccount "payload-processing-e2e-extmod-a-5c1e58" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-5c1e58: serviceaccount "payload-processing-e2e-extmod-a-5c1e58" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-5c1e58

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-812032-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-812032-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

models-as-a-service

kubelet

payload-processing-external-model-646f5f8dd4-s2qzd

Killing

Stopping container payload-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-2a8c48-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-3da894-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-2a8c48-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-3da894-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

ProbeError

Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

Unhealthy

Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

ProbeError

Readiness probe error: Get "http://10.134.0.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x8)

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

ProbeError

Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Unhealthy

Readiness probe failed: Get "http://10.134.0.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x8)

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

Unhealthy

Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused
(x2)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Unhealthy

Readiness probe failed: Get "https://10.134.0.38:8000/ready": context deadline exceeded
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-be2a74-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-be2a74-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-023307-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-023307-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-a558a6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-a558a6

openshift-ingress

deployment-controller

e2e-extmod-b-a558a6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-a558a6-openshift-default-57d8959bdf from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-a558a6-openshift-default-57d8959bdf

SuccessfulCreate

Created pod: e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

openshift-ingress

multus

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

AddedInterface

Add eth0 [10.133.0.75/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-a558a6: serviceaccount "payload-processing-e2e-extmod-b-a558a6" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-a558a6-5fddcb598

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-a558a6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-a558a6-5fddcb598 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-a558a6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-a558a6-5fddcb598

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-a558a6-5fddcb598-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-a558a6: serviceaccount "payload-processing-e2e-extmod-b-a558a6" not found

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm

AddedInterface

Add eth0 [10.134.0.84/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

openshift-ingress

deployment-controller

e2e-worker-w3-60ed4a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-60ed4a-openshift-default-76f5998f8c from 5 to 10

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-60ed4a-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

openshift-ingress

replicaset-controller

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c

SuccessfulCreate

Created pod: e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

AddedInterface

Add eth0 [10.134.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

Created

Container created

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

AddedInterface

Add eth0 [10.134.0.87/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

Started

Container started

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

Created

Container created

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

Started

Container started

openshift-ingress

multus

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-60ed4a-fd7969596-zfk2k

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5c1e58-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5c1e58-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-60ed4a-5b98579987-w25tt

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

Unhealthy

Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

ProbeError

Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

ProbeError

Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-be2a74-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-be2a74-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

Unhealthy

Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

Unhealthy

Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

ProbeError

Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-023307-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-023307-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

ProbeError

Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

Unhealthy

Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

ProbeError

Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

Unhealthy

Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5c1e58-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5c1e58-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-f5qwx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4b6d9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-cpgbr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-n4gcq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jlz4q

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-jsjnl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-6qgmp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

ProbeError

Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

Unhealthy

Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6dd194-5794949964-bd72w

Killing

Stopping container payload-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-a558a6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-a558a6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh

Killing

Stopping container payload-pre-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-a558a6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

models-as-a-service

replicaset-controller

payload-processing-external-model-8474b49dbc

FailedCreate

Error creating: pods "payload-processing-external-model-8474b49dbc-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-a558a6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7476dd8d5d from 0 to 1

openshift-ingress

kubelet

e2e-extmod-a-6dd194-openshift-default-6f76f45889-9rnf2

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

models-as-a-service

replicaset-controller

payload-processing-external-model-8474b49dbc

SuccessfulCreate

Created pod: payload-processing-external-model-8474b49dbc-pxhg8

models-as-a-service

replicaset-controller

payload-processing-external-model-7476dd8d5d

SuccessfulCreate

Created pod: payload-processing-external-model-7476dd8d5d-n9wd4

models-as-a-service

multus

payload-processing-external-model-8474b49dbc-pxhg8

AddedInterface

Add eth0 [10.134.0.92/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-pxhg8

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-pxhg8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-pxhg8

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-7476dd8d5d-n9wd4

Started

Container started

models-as-a-service

multus

payload-processing-external-model-7476dd8d5d-n9wd4

AddedInterface

Add eth0 [10.134.0.93/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-7476dd8d5d-n9wd4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-7476dd8d5d-n9wd4

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-812032-6c4759db6c-tqhmp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-8474b49dbc

SuccessfulDelete

Deleted pod: payload-processing-external-model-8474b49dbc-pxhg8

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-pxhg8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-b-812032-openshift-default-9f8549b85-fqs6x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-29d3ee-6b69556f7b-798b5

Killing

Stopping container payload-processing
(x2)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-8474b49dbc from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-8474b49dbc

SuccessfulCreate

Created pod: payload-processing-external-model-8474b49dbc-dvj9v

models-as-a-service

multus

payload-processing-external-model-8474b49dbc-dvj9v

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-dvj9v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-dvj9v

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-dvj9v

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6d5d9997cc from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-6d5d9997cc

SuccessfulCreate

Created pod: payload-processing-external-model-6d5d9997cc-qr9gm

models-as-a-service

replicaset-controller

payload-processing-external-model-8474b49dbc

SuccessfulDelete

Deleted pod: payload-processing-external-model-8474b49dbc-dvj9v
(x2)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-8474b49dbc from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-8474b49dbc-dvj9v

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-6d5d9997cc-qr9gm

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-6d5d9997cc-qr9gm

Created

Container created

models-as-a-service

multus

payload-processing-external-model-6d5d9997cc-qr9gm

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6d5d9997cc-qr9gm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-9699b4dc9

SuccessfulCreate

Created pod: payload-processing-external-model-9699b4dc9-g954t

models-as-a-service

kubelet

payload-processing-external-model-6d5d9997cc-qr9gm

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-9699b4dc9 from 0 to 1
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-29d3ee-6b9d8774f9-67lmg

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

multus

payload-processing-external-model-9699b4dc9-g954t

AddedInterface

Add eth0 [10.133.0.79/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-9699b4dc9-g954t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-6d5d9997cc

SuccessfulDelete

Deleted pod: payload-processing-external-model-6d5d9997cc-qr9gm

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-6d5d9997cc from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-9699b4dc9-g954t

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-9699b4dc9-g954t

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-29d3ee-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w1-29d3ee-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-29d3ee-openshift-default-844cdd95d7 from 2 to 3

openshift-ingress

replicaset-controller

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7

SuccessfulCreate

Created pod: e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

Started

Container started

openshift-ingress

multus

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-3da894-openshift-default-695bd64b77-m5dqp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

Killing

Stopping container payload-pre-processing
(x6)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w1-29d3ee-openshift-default-dockercfg-xktsc); attempting to pull the image may not succeed.

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-be2a74-6c9d9fb9d8-77qxg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-xsx6x

Killing

Stopping container istio-proxy

models-as-a-service

kubelet

payload-processing-external-model-7476dd8d5d-n9wd4

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-7476dd8d5d

SuccessfulDelete

Deleted pod: payload-processing-external-model-7476dd8d5d-n9wd4

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-4fgkl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-7476dd8d5d from 1 to 0

openshift-ingress

kubelet

e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-a-2a8c48-openshift-default-5d7b54765-sptdv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-9699b4dc9-g954t

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-023307-8c4d6b74d-k4hsb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-2a8c48-76c84d49c5-gr6h6

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.68:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5c1e58-674ddfdf5d-wmncg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5c1e58-65d4b77567-4b5sn

Killing

Stopping container payload-pre-processing
(x8)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-5b4gn

Unhealthy

Readiness probe failed: Get "https://10.134.0.38:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-be2a74-6c4f95bb79-rwc5t

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.74:50052" within 1s: context deadline exceeded
(x2)

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceNotReady

LLMInferenceService [premium-simulated-simulated-premium] is no longer Ready because of: MainWorkloadReady, WorkloadsReady
(x3)

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Unhealthy

Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

ProbeError

Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-023307-openshift-default-645c679bbf-b4xvc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

ProbeError

Readiness probe error: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

Unhealthy

Readiness probe failed: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

ProbeError

Readiness probe error: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Unhealthy

Readiness probe failed: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-a-5c1e58-openshift-default-5cdb84f5bf-xcggn

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w2-016e6c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-016e6c-openshift-default-644bc689bc from 8 to 10

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-016e6c-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w2-016e6c-openshift-default-644bc689bc

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

Started

Container started

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

AddedInterface

Add eth0 [10.134.0.106/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

Started

Container started
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-016e6c-7c649d4899-dw9wc

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-016e6c-68fd9678fb-rcfvh

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-a558a6-5fddcb598-c5drn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-a558a6-6f8689669f-5q5cm

Killing

Stopping container payload-pre-processing
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

Unhealthy

Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

ProbeError

Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

ProbeError

Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

Unhealthy

Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-j46gr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-4v9zd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nwrwl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-bdh8p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-5tp6v

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-nmcr9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-df3be8-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

AddedInterface

Add eth0 [10.133.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

AddedInterface

Add eth0 [10.133.0.82/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

Created

Container created

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

openshift-ingress

deployment-controller

e2e-worker-w0-df3be8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-df3be8-openshift-default-854766f5d9 from 5 to 10

openshift-ingress

multus

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

AddedInterface

Add eth0 [10.134.0.108/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-df3be8-openshift-default-854766f5d9

SuccessfulCreate

Created pod: e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-df3be8-b96f6dc59-ggjrv

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-df3be8-5f967dd678-2dz9b

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

ProbeError

Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

Unhealthy

Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

Unhealthy

Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

ProbeError

Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

Unhealthy

Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

Unhealthy

Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

ProbeError

Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

ProbeError

Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1f6ce4-7f9dfb9d44-k7tnn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cv2v5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-gc9tz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kclqr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-f85ks

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mv8cz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-fxwv9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

AddedInterface

Add eth0 [10.134.0.110/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-1f6ce4-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c

SuccessfulCreate

Created pod: e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

openshift-ingress

deployment-controller

e2e-worker-w4-1f6ce4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c from 2 to 3
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1f6ce4-657d7c9887-gtsfm

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

ProbeError

Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

Unhealthy

Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-czm6g

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-qh5bb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-rxf9k

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-rxf9k

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-788ls

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-xgjgh

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 10 to 6

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-g5rgr

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-xgjgh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-788ls

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-g5rgr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-8j245

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8j245

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8j245

Unhealthy

Readiness probe failed: Get "http://10.132.0.68:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8j245

ProbeError

Readiness probe error: Get "http://10.132.0.68:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 6 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-r7fn5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-5qthm

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-5qthm

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-r7fn5

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 4 to 2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-7wfxn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-lhm7v

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-lhm7v

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-7wfxn

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-74854d8ccf from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-74854d8ccf-mvvkh

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-74854d8ccf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-74854d8ccf-mvvkh