Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt to ip-10-0-138-179.us-west-2.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-d5rzd

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-d5rzd to ip-10-0-128-142.us-west-2.compute.internal

ai-tenants

e2e-extmod-a-b1047d

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-b1047d: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-1225f3

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-1225f3: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-cae0ab

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-cae0ab: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-d057d7

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-d057d7: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-a96a99

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-a96a99: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w1-1baf54

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w1-1baf54: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-00cfe6

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-00cfe6: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-9f7183

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-9f7183: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-b2eadb

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-b2eadb: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-97zxq

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-97zxq to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9 to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj to ip-10-0-138-179.us-west-2.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-djfnz

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-djfnz to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978 to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw to ip-10-0-138-179.us-west-2.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-5d2lj

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-5d2lj to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-7w5l8 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-78bvs

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-78bvs to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-6c786f9b57m5fpl

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6c786f9b57m5fpl to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-449zx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-449zx to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-2lkmv to ip-10-0-128-142.us-west-2.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm to ip-10-0-131-51.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-765779c7c6-wn22v

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-765779c7c6-wn22v to ip-10-0-138-179.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-7789777d6-l8vds

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7789777d6-l8vds to ip-10-0-138-179.us-west-2.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-jf7b5

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-jf7b5 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

kube-auth-proxy-54cfc47d9b-6g2fd

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-54cfc47d9b-6g2fd to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

kube-auth-proxy-54cfc47d9b-5tzlw

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-54cfc47d9b-5tzlw to ip-10-0-138-179.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-7b4675d6f7-sq4wn

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7b4675d6f7-sq4wn to ip-10-0-131-51.us-west-2.compute.internal

kuadrant-system

authorino-56c6c9ddd5-nq9z2

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-nq9z2 to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gfwmm to ip-10-0-138-179.us-west-2.compute.internal

openshift-jobset-operator

jobset-controller-manager-55d4cc59f-nt2bd

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-55d4cc59f-nt2bd to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2jpkz to ip-10-0-131-51.us-west-2.compute.internal

opendatahub

ai-gateway-controller-5d8dddb676-fjq9d

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-fjq9d to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

router-default-f5c95ffcf-hl6wf

Scheduled

Successfully assigned openshift-ingress/router-default-f5c95ffcf-hl6wf to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

router-default-f5c95ffcf-hl6wf

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-f5c95ffcf-hl6wf

FailedScheduling

no nodes available to schedule pods

openshift-ingress

payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld to ip-10-0-138-179.us-west-2.compute.internal

kuadrant-system

authorino-57f7f8659f-89qwq

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-89qwq to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6 to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94 to ip-10-0-128-142.us-west-2.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-6f4h4

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-6f4h4 to ip-10-0-131-51.us-west-2.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-mjdhk

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-mjdhk to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst to ip-10-0-131-51.us-west-2.compute.internal

kuadrant-system

authorino-6f6944bcdd-ghspq

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-ghspq to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6 to ip-10-0-138-179.us-west-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-5bdd5c57b4-v842j to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

payload-processing-7bb9489676-6prq8

Scheduled

Successfully assigned openshift-ingress/payload-processing-7bb9489676-6prq8 to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-8xfgm to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp to ip-10-0-131-51.us-west-2.compute.internal

kuadrant-system

authorino-7d7745c5b5-wcgxs

Scheduled

Successfully assigned kuadrant-system/authorino-7d7745c5b5-wcgxs to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx to ip-10-0-138-179.us-west-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2 to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx to ip-10-0-138-179.us-west-2.compute.internal

kuadrant-system

authorino-8944767cb-69krk

Scheduled

Successfully assigned kuadrant-system/authorino-8944767cb-69krk to ip-10-0-131-51.us-west-2.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-n6pxb

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-n6pxb to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv to ip-10-0-138-179.us-west-2.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8 to ip-10-0-138-179.us-west-2.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-flqz4

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-flqz4 to ip-10-0-131-51.us-west-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-5bdd5c57b4-bgs65 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-p567b

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-p567b to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-jfjx2

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-jfjx2 to ip-10-0-131-51.us-west-2.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-gqxkr

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-gqxkr to ip-10-0-131-51.us-west-2.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lcrv9 to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6 to ip-10-0-128-142.us-west-2.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-4sws9

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-4sws9 to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-7454465567-qvplm to ip-10-0-138-179.us-west-2.compute.internal

ai-tenants

e2e-extmod-a-5520b3

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-5520b3: MaasTenantConfig is not available to restore revocation operands

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv to ip-10-0-131-51.us-west-2.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-f6w52

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-f6w52 to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-wbbff to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-t9ljt to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-qch24

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-qch24 to ip-10-0-138-179.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-5dddfc6875-9n4rm

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5dddfc6875-9n4rm to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-q6lf6 to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-m7jld to ip-10-0-128-142.us-west-2.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-jkdvh to ip-10-0-131-51.us-west-2.compute.internal

opendatahub

test-curl-7633-ac7032

Scheduled

Successfully assigned opendatahub/test-curl-7633-ac7032 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6 to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt to ip-10-0-128-142.us-west-2.compute.internal

opendatahub

test-curl-7633-346153

Scheduled

Successfully assigned opendatahub/test-curl-7633-346153 to ip-10-0-138-179.us-west-2.compute.internal

opendatahub

odh-model-controller-769875db56-pxmvp

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-pxmvp to ip-10-0-128-142.us-west-2.compute.internal

opendatahub

model-serving-api-75bd89fdfc-znvc4

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-znvc4 to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s to ip-10-0-138-179.us-west-2.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-6qnlj

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-6qnlj to ip-10-0-138-179.us-west-2.compute.internal

opendatahub

maas-controller-84564f7548-5s5tr

Scheduled

Successfully assigned opendatahub/maas-controller-84564f7548-5s5tr to ip-10-0-128-142.us-west-2.compute.internal

opendatahub

maas-controller-557bcb748c-j5ss5

Scheduled

Successfully assigned opendatahub/maas-controller-557bcb748c-j5ss5 to ip-10-0-128-142.us-west-2.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-dv8d8

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-dv8d8 to ip-10-0-128-142.us-west-2.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-jhr6j

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-jhr6j to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv to ip-10-0-128-142.us-west-2.compute.internal

ai-tenants

e2e-extmod-a-6db2a7

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-6db2a7: MaasTenantConfig is not available to restore revocation operands

opendatahub

kserve-controller-manager-776cc66d47-9vstw

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-9vstw to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-gh8zv to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-fn6tt to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-dpdg9 to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6 to ip-10-0-138-179.us-west-2.compute.internal

kuadrant-system

authorino-cc745859-ntsd5

Scheduled

Successfully assigned kuadrant-system/authorino-cc745859-ntsd5 to ip-10-0-131-51.us-west-2.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-pbz49

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-pbz49 to ip-10-0-138-179.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-6fddb87f65-hxqhz

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6fddb87f65-hxqhz to ip-10-0-138-179.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-98jdc to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s to ip-10-0-138-179.us-west-2.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-n977w

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-n977w to ip-10-0-128-142.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5db77fbd9f-b7kv4 to ip-10-0-138-179.us-west-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-lzpmm

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-lzpmm to ip-10-0-138-179.us-west-2.compute.internal

opendatahub

ai-gateway-controller-74b5554cc8-xwm2f

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-74b5554cc8-xwm2f to ip-10-0-131-51.us-west-2.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-f5c95ffcf from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-f5c95ffcf

SuccessfulCreate

Created pod: router-default-f5c95ffcf-hl6wf
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-f5c95ffcf-hl6wf

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-f5c95ffcf-hl6wf

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-f5c95ffcf-hl6wf

AddedInterface

Add eth0 [10.132.0.10/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-f5c95ffcf-hl6wf

Unhealthy

Startup probe failed: HTTP probe failed with statuscode: 500

openshift-ingress

kubelet

router-default-f5c95ffcf-hl6wf

Created

Container created

openshift-ingress

kubelet

router-default-f5c95ffcf-hl6wf

Started

Container started

openshift-ingress

kubelet

router-default-f5c95ffcf-hl6wf

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

router-default-f5c95ffcf-hl6wf

ProbeError

Startup probe error: HTTP probe failed with statuscode: 500 body: [-]backend-proxy-http failed: reason withheld [-]has-synced failed: reason withheld [+]process-running ok healthz check failed

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-p567b

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-p567b

AddedInterface

Add eth0 [10.134.0.20/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-p567b

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-p567b

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.557s (4.557s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-p567b

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-p567b

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-p567b

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-p567b

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 0 to 1

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-wbbff

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-7454465567 from 0 to 1

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-7454465567

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 6.511s (6.511s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.301s (9.301s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

Created

Container created

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

Started

Container started

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

Unhealthy

Startup probe failed: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-7454465567-qvplm

ProbeError

Startup probe error: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-5bdd5c57b4

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-5bdd5c57b4

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-5bdd5c57b4

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-5bdd5c57b4-v842j

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-5bdd5c57b4 from 0 to 3

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

NeedsReinstall

calculated deployment install is bad
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.679s (3.679s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.267s (4.267s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.822s (3.822s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 980ms (980ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 851ms (851ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x2)

opendatahub-operator-system

operator-lifecycle-manager

install-btm4t

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-v842j

Started

Container started
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 835ms (835ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5bdd5c57b4-bgs65

Started

Container started

opendatahub-operator-system

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2_faff9907-e9b7-4916-8530-d748f9eff93d

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-5bdd5c57b4-bl6n2_faff9907-e9b7-4916-8530-d748f9eff93d became leader
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.
(x3)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-4sws9

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-4sws9

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

kuadrant-system

multus

authorino-operator-7b7bb4bc77-4sws9

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-4sws9

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-4sws9

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 4.054s (4.054s including waiting). Image size: 167738465 bytes.

kuadrant-system

authorino-operator-7b7bb4bc77-4sws9_555702b8-a3c0-491f-80fa-0eac13eb0ca8

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-4sws9_555702b8-a3c0-491f-80fa-0eac13eb0ca8 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-4sws9

Started

Container started

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-lzpmm

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-lzpmm

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-lzpmm

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-n977w

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-n977w

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-n977w

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-5d2lj

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

FailedCreate

Error creating: pods "cert-manager-webhook-5dbbbf75cc-" is forbidden: error looking up service account cert-manager/cert-manager-webhook: serviceaccount "cert-manager-webhook" not found

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5d2lj

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-5d2lj

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-lzpmm

Created

Container created

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-6qnlj

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-6qnlj

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-lzpmm

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 5.589s (5.589s including waiting). Image size: 424244516 bytes.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-lzpmm

Started

Container started

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-n977w

Started

Container started

kuadrant-system

limitador-operator-controller-manager-5778bb4449-n977w_0859a91b-60f2-4bc0-b6c6-6e2f0467c9ee

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-n977w_0859a91b-60f2-4bc0-b6c6-6e2f0467c9ee became leader

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-n977w

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-n977w

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 2.966s (2.966s including waiting). Image size: 177967631 bytes.

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-lzpmm_3f645702-0a32-4acf-95b3-e657affb4196

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-lzpmm_3f645702-0a32-4acf-95b3-e657affb4196 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-6qnlj

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"
(x11)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-pbz49

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-djfnz

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-6qnlj

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 3.496s (3.497s including waiting). Image size: 190804149 bytes.

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-6qnlj

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-6qnlj

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5d2lj

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5d2lj

Created

Container created

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-5d2lj

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 5.507s (5.507s including waiting). Image size: 309650901 bytes.

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pbz49

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-pbz49

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-djfnz

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

dns-operator-controller-manager-8557cbdb84-6qnlj_4a1b5400-88fc-4d1b-8d2d-70a822deceea

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-6qnlj_4a1b5400-88fc-4d1b-8d2d-70a822deceea became leader

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-djfnz

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-97zxq

cert-manager

kubelet

cert-manager-597b95cb84-97zxq

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

cert-manager

multus

cert-manager-597b95cb84-97zxq

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-djfnz

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-djfnz

Created

Container created

cert-manager

kubelet

cert-manager-597b95cb84-97zxq

Created

Container created

cert-manager

kubelet

cert-manager-597b95cb84-97zxq

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-djfnz

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 4.146s (4.146s including waiting). Image size: 309650901 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pbz49

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pbz49

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 6.21s (6.21s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pbz49

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-flqz4

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-flqz4

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-flqz4

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-flqz4

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-flqz4

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-flqz4

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 4.191s (4.191s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-flqz4_48aa7c7e-35e1-4bb8-84a7-7c8e4f31ef53 became leader

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-ingress

replicaset-controller

kube-auth-proxy-54cfc47d9b

SuccessfulCreate

Created pod: kube-auth-proxy-54cfc47d9b-5tzlw

openshift-ingress

replicaset-controller

kube-auth-proxy-54cfc47d9b

SuccessfulCreate

Created pod: kube-auth-proxy-54cfc47d9b-6g2fd

openshift-ingress

multus

kube-auth-proxy-54cfc47d9b-6g2fd

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-54cfc47d9b from 0 to 2

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-6c786f9b57

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-6c786f9b57m5fpl

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-6c786f9b57 from 0 to 1

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6c786f9b57m5fpl

Started

Container started

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6c786f9b57m5fpl

Created

Container created

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6c786f9b57m5fpl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

kube-auth-proxy-54cfc47d9b-5tzlw

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-5tzlw

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-f22sj"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-vzvdm"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-6g2fd

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-6c786f9b57m5fpl

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-f6w52

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

kuadrant-system

multus

limitador-limitador-74dfcfb894-f6w52

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-f6w52

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-6g2fd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.276s (3.276s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-6g2fd

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-89qwq

kuadrant-system

kubelet

authorino-57f7f8659f-89qwq

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-6g2fd

Created

Container created

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

kuadrant-system

multus

authorino-57f7f8659f-89qwq

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-5tzlw

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-5tzlw

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-54cfc47d9b-5tzlw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3s (3s including waiting). Image size: 188904677 bytes.

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-nq9z2

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-55d4cc59f

SuccessfulCreate

Created pod: jobset-controller-manager-55d4cc59f-nt2bd

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-55d4cc59f from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-jf7b5

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

multus

authorino-56c6c9ddd5-nq9z2

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-jf7b5

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

openshift-jobset-operator

multus

jobset-controller-manager-55d4cc59f-nt2bd

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-controller-manager-55d4cc59f-nt2bd

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

kuadrant-system

kubelet

authorino-56c6c9ddd5-nq9z2

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

kuadrant-system

multus

limitador-limitador-7c9b87d574-jf7b5

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-f6w52

Started

Container started

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-f6w52

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 2.501s (2.501s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-f6w52

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-89qwq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.266s (3.266s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-nq9z2

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-89qwq

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-nq9z2

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-89qwq

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-nq9z2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.15s (2.15s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-jf7b5

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-jf7b5

Started

Container started

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-jf7b5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 2.519s (2.519s including waiting). Image size: 133379307 bytes.

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-89qwq

kuadrant-system

kubelet

authorino-57f7f8659f-89qwq

Killing

Stopping container authorino

openshift-jobset-operator

kubelet

jobset-controller-manager-55d4cc59f-nt2bd

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 4.514s (4.514s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-55d4cc59f-nt2bd

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-55d4cc59f-nt2bd

Started

Container started

openshift-jobset-operator

jobset-controller-manager-55d4cc59f-nt2bd_fca4391c-2ca2-4b14-b0f2-cf86f6590181

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-55d4cc59f-nt2bd_fca4391c-2ca2-4b14-b0f2-cf86f6590181 became leader

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-f6w52

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-f6w52

Killing

Stopping container limitador

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-d5rzd

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-d5rzd

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-d5rzd

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-d5rzd

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-d5rzd

Started

Container started

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-d5rzd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 3.731s (3.731s including waiting). Image size: 195729000 bytes.
(x3)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-557bcb748c from 0 to 1

opendatahub

multus

maas-controller-84564f7548-5s5tr

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-84564f7548 from 0 to 1

opendatahub

replicaset-controller

maas-controller-84564f7548

SuccessfulCreate

Created pod: maas-controller-84564f7548-5s5tr

opendatahub

replicaset-controller

maas-controller-557bcb748c

SuccessfulCreate

Created pod: maas-controller-557bcb748c-j5ss5

opendatahub

kserve-module-controller-manager-7566cf5fc8-d5rzd_a6ecf9fb-50a7-455d-ae8a-9d8438bcfb71

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-d5rzd_a6ecf9fb-50a7-455d-ae8a-9d8438bcfb71 became leader

opendatahub

multus

maas-controller-557bcb748c-j5ss5

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-84564f7548-5s5tr

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:8f56f406b6bbaa864395476ab7f22098ab75da4f3382bb2d7852b12111924f3d"

opendatahub

kubelet

maas-controller-84564f7548-5s5tr

Started

Container started

opendatahub

kubelet

maas-controller-557bcb748c-j5ss5

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:8f56f406b6bbaa864395476ab7f22098ab75da4f3382bb2d7852b12111924f3d" in 6.796s (6.796s including waiting). Image size: 200103445 bytes.

opendatahub

kubelet

maas-controller-84564f7548-5s5tr

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:8f56f406b6bbaa864395476ab7f22098ab75da4f3382bb2d7852b12111924f3d" in 7.362s (7.362s including waiting). Image size: 200103445 bytes.

opendatahub

kubelet

maas-controller-84564f7548-5s5tr

Created

Container created

opendatahub

maas-controller-84564f7548-5s5tr_58ea6de2-8b9a-4533-9855-95f82f4d56e3

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-84564f7548-5s5tr_58ea6de2-8b9a-4533-9855-95f82f4d56e3 became leader

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

FailedMount

MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

FailedMount

MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-84564f7548 from 1 to 0

opendatahub

kubelet

maas-controller-84564f7548-5s5tr

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-84564f7548

SuccessfulDelete

Deleted pod: maas-controller-84564f7548-5s5tr

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-lcrv9

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.918s (13.918s including waiting). Image size: 1089502687 bytes.

opendatahub

maas-controller-557bcb748c-j5ss5_fff47e4f-3493-4bac-b101-cd858224d6fd

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-557bcb748c-j5ss5_fff47e4f-3493-4bac-b101-cd858224d6fd became leader

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-gfwmm

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-9vstw

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.963s (4.963s including waiting). Image size: 185476477 bytes.

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-znvc4

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-dv8d8

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

kubelet

kserve-controller-manager-776cc66d47-9vstw

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

multus

kserve-controller-manager-776cc66d47-9vstw

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

multus

odh-model-controller-769875db56-pxmvp

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-dv8d8

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-pxmvp

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-dv8d8

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-znvc4

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

multus

model-serving-api-75bd89fdfc-znvc4

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gfwmm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

opendatahub

kubelet

odh-model-controller-769875db56-pxmvp

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-p567b

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-p567b

Killing

Stopping container discovery

opendatahub

kubelet

model-serving-api-75bd89fdfc-znvc4

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-dv8d8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 5.202s (5.202s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-9vstw

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-9vstw

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 5.321s (5.321s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-9vstw

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-znvc4

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-dv8d8

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-dv8d8

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-znvc4

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 5.087s (5.087s including waiting). Image size: 184925761 bytes.

opendatahub

kserve-controller-manager-776cc66d47-9vstw_8bfd5803-c153-46f3-b463-dee6d2b26269

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-9vstw_8bfd5803-c153-46f3-b463-dee6d2b26269 became leader

opendatahub

kubelet

odh-model-controller-769875db56-pxmvp

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-pxmvp

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-pxmvp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 6.574s (6.574s including waiting). Image size: 236755567 bytes.

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-ghspq
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-7d7745c5b5 from 0 to 1

kuadrant-system

replicaset-controller

authorino-7d7745c5b5

SuccessfulCreate

Created pod: authorino-7d7745c5b5-wcgxs

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

opendatahub

llmisvc-controller-manager-869d5dcc5d-dv8d8_c044f085-5abd-47fa-9f46-0c2c0d569a3f

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-dv8d8_c044f085-5abd-47fa-9f46-0c2c0d569a3f became leader

kuadrant-system

multus

authorino-6f6944bcdd-ghspq

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

odh-model-controller-769875db56-pxmvp_0ad10c76-3ae2-48a3-8a69-89250a3ac0b4

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-pxmvp_0ad10c76-3ae2-48a3-8a69-89250a3ac0b4 became leader

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-ghspq

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-cc745859 from 0 to 1

kuadrant-system

replicaset-controller

authorino-cc745859

SuccessfulCreate

Created pod: authorino-cc745859-ntsd5

kuadrant-system

multus

authorino-7d7745c5b5-wcgxs

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-7d7745c5b5-wcgxs

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-6f6944bcdd-ghspq

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-cc745859-ntsd5

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-cc745859-ntsd5

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-7d7745c5b5

SuccessfulDelete

Deleted pod: authorino-7d7745c5b5-wcgxs

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-7d7745c5b5 from 1 to 0
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

ProbeError

Startup probe error: Get "http://10.133.0.20:15021/healthz/ready": dial tcp 10.133.0.20:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Unhealthy

Startup probe failed: Get "http://10.133.0.20:15021/healthz/ready": dial tcp 10.133.0.20:15021: connect: connection refused

kuadrant-system

kubelet

authorino-7d7745c5b5-wcgxs

Started

Container started

kuadrant-system

kubelet

authorino-cc745859-ntsd5

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-ghspq

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-ghspq

Started

Container started

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-nq9z2

kuadrant-system

kubelet

authorino-6f6944bcdd-ghspq

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-56c6c9ddd5-nq9z2

Killing

Stopping container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

kubelet

authorino-cc745859-ntsd5

Started

Container started

kuadrant-system

kubelet

authorino-7d7745c5b5-wcgxs

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 4.649s (4.649s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-7d7745c5b5-wcgxs

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-ghspq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 5.417s (5.417s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-7d7745c5b5-wcgxs

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-cc745859-ntsd5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 4.627s (4.627s including waiting). Image size: 204096810 bytes.
(x5)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-jhr6j

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-jhr6j

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-6466c554d8-jhr6j

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-jhr6j

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-jhr6j

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 837ms (837ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-jhr6j

Created

Container created
(x2)

opendatahub

kubelet

maas-controller-557bcb748c-j5ss5

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:8f56f406b6bbaa864395476ab7f22098ab75da4f3382bb2d7852b12111924f3d"

opendatahub

kubelet

maas-controller-557bcb748c-j5ss5

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:8f56f406b6bbaa864395476ab7f22098ab75da4f3382bb2d7852b12111924f3d" in 1.141s (1.141s including waiting). Image size: 200103445 bytes.
(x2)

opendatahub

kubelet

maas-controller-557bcb748c-j5ss5

Created

Container created
(x2)

opendatahub

kubelet

maas-controller-557bcb748c-j5ss5

Started

Container started

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-dv8d8

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-dv8d8

Killing

Stopping container manager

opendatahub

multus

ai-gateway-controller-5d8dddb676-fjq9d

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-fjq9d

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-fjq9d

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-fjq9d

Created

Container created

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-fjq9d

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-fjq9d

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 2.564s (2.564s including waiting). Image size: 184136103 bytes.

opendatahub

ai-gateway-controller-5d8dddb676-fjq9d_dcfd7d0a-f7a5-4660-b456-e1e776ff20f6

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-fjq9d_dcfd7d0a-f7a5-4660-b456-e1e776ff20f6 became leader

opendatahub

maas-controller-557bcb748c-j5ss5_c814cde4-9f7d-48b0-86dd-e4127c7cd235

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-557bcb748c-j5ss5_c814cde4-9f7d-48b0-86dd-e4127c7cd235 became leader
(x7)

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

FailedCreate

Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-jfjx2
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

opendatahub

llmisvc-controller-manager-6466c554d8-jhr6j_44d15f54-ece2-4a5e-998b-ce3aff1c6fc9

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-jhr6j_44d15f54-ece2-4a5e-998b-ce3aff1c6fc9 became leader

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

SuccessfulCreate

Created pod: payload-processing-7bb9489676-6prq8

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-7bb9489676 from 0 to 1

openshift-ingress

kubelet

payload-processing-7bb9489676-6prq8

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-7bb9489676-6prq8

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-7b98bb678c-jfjx2

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-jfjx2

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-7bb9489676-6prq8

Created

Container created

openshift-ingress

kubelet

payload-processing-7bb9489676-6prq8

Started

Container started

openshift-ingress

kubelet

payload-processing-7bb9489676-6prq8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.993s (2.993s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-jfjx2

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.07s (3.07s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-jfjx2

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-jfjx2

Started

Container started

opendatahub

replicaset-controller

ai-gateway-controller-74b5554cc8

SuccessfulCreate

Created pod: ai-gateway-controller-74b5554cc8-xwm2f

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-74b5554cc8 from 0 to 1

opendatahub

kubelet

ai-gateway-controller-74b5554cc8-xwm2f

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

multus

ai-gateway-controller-74b5554cc8-xwm2f

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-74b5554cc8-xwm2f

Started

Container started

opendatahub

kubelet

ai-gateway-controller-74b5554cc8-xwm2f

Created

Container created

opendatahub

kubelet

ai-gateway-controller-74b5554cc8-xwm2f

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 911ms (911ms including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-fjq9d

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-fjq9d

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

ai-gateway-controller-74b5554cc8-xwm2f_c8081b6e-d4bb-42d2-a9ea-63be4866e98a

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-74b5554cc8-xwm2f_c8081b6e-d4bb-42d2-a9ea-63be4866e98a became leader

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf

Created

Container created

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-k6pdf

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.744s (2.744s including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-gqxkr

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gqxkr

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-gqxkr

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gqxkr

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 588ms (589ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gqxkr

Started

Container started

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gqxkr

Created

Container created

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-n6pxb

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-n6pxb

Started

Container started

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-n6pxb

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-n6pxb

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-n6pxb

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 647ms (647ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-n6pxb

Created

Container created

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-6f4h4

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-6f4h4

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6f4h4

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6f4h4

Started

Container started

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6f4h4

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 661ms (661ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-6f4h4

Created

Container created

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm

Created

Container created

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 593ms (593ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mlswm

Started

Container started

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-mjdhk

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

kubelet

echo-simulator-kserve-64b6f8f96c-mjdhk

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

multus

echo-simulator-kserve-64b6f8f96c-mjdhk

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

kubelet

echo-simulator-kserve-64b6f8f96c-mjdhk

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.727s (2.727s including waiting). Image size: 78208878 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-mjdhk

Created

Container created

llm

kubelet

echo-simulator-kserve-64b6f8f96c-mjdhk

Started

Container started

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-qch24

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 1 to 2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-qch24

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-qch24

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-qch24

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-qch24

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8

Started

Container started

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.976s (2.976s including waiting). Image size: 98346788 bytes.

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-r46c8

Created

Container created

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv

Created

Container created

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 566ms (566ms including waiting). Image size: 98346788 bytes.

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-t4rgv

Started

Container started
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-qch24

Unhealthy

Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-qch24

ProbeError

Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused body:

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 2 to 4

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

Started

Container started

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

kuadrant-system

replicaset-controller

authorino-8944767cb

SuccessfulCreate

Created pod: authorino-8944767cb-69krk

kuadrant-system

multus

authorino-8944767cb-69krk

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-8944767cb-69krk

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-8944767cb-69krk

Created

Container created

kuadrant-system

kubelet

authorino-8944767cb-69krk

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.702s (1.702s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-8944767cb-69krk

Started

Container started

kuadrant-system

kubelet

authorino-cc745859-ntsd5

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-cc745859

SuccessfulDelete

Deleted pod: authorino-cc745859-ntsd5
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-cc745859 from 1 to 0
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.529s (8.529s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 5; reason:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-m7jld

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

Unhealthy

Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

ProbeError

Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-9f7183

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-9f7183

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-00cfe6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-00cfe6

openshift-ingress

replicaset-controller

e2e-worker-w2-00cfe6-openshift-default-64579789c7

SuccessfulCreate

Created pod: e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

openshift-ingress

deployment-controller

e2e-worker-w2-00cfe6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-00cfe6-openshift-default-64579789c7 from 0 to 1

openshift-ingress

multus

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w3-9f7183-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-9f7183-openshift-default-646bd94655 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w3-9f7183-openshift-default-646bd94655

SuccessfulCreate

Created pod: e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

openshift-ingress

multus

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w1-1baf54

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-1baf54

openshift-ingress

deployment-controller

e2e-worker-w1-1baf54-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-1baf54-openshift-default-84dc6cddcc from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-9f7183

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-9f7183-764969574c from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc

SuccessfulCreate

Created pod: e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

openshift-ingress

multus

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-9f7183

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-9f7183-84b59f48b4 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-9f7183-84b59f48b4

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-9f7183-84b59f48b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-9f7183: serviceaccount "payload-processing-e2e-worker-w3-9f7183" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-9f7183-764969574c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-9f7183-764969574c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-9f7183-764969574c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-9f7183: serviceaccount "payload-processing-e2e-worker-w3-9f7183" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-9f7183-84b59f48b4

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt

Started

Container started

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-b2eadb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-b2eadb

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-a96a99

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-a96a99

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.897s (2.897s including waiting). Image size: 125353963 bytes.

openshift-ingress

multus

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulCreate

Created pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

openshift-ingress

deployment-controller

e2e-worker-w0-a96a99-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-a96a99-openshift-default-6ccb446cf5 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

Created

Container created

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-00cfe6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-00cfe6: serviceaccount "payload-processing-e2e-worker-w2-00cfe6" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-00cfe6: serviceaccount "payload-processing-e2e-worker-w2-00cfe6" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-00cfe6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-00cfe6-84444cfdcd from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

openshift-ingress

multus

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

Created

Container created
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-a96a99-757ff68588

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-a96a99-757ff68588-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-a96a99: serviceaccount "payload-processing-e2e-worker-w0-a96a99" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-a96a99-6576946687

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-a96a99-6576946687-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-a96a99: serviceaccount "payload-processing-e2e-worker-w0-a96a99" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-a96a99-6576946687

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-a96a99

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-a96a99-6576946687 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-a96a99

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-a96a99-757ff68588 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-a96a99-757ff68588

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-00cfe6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-00cfe6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w1-1baf54

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w1-1baf54-666476dc6b from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

ProbeError

Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused body:
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-1baf54-666476dc6b

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w1-1baf54-666476dc6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-1baf54: serviceaccount "payload-processing-e2e-worker-w1-1baf54" not found
(x5)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

Unhealthy

Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-1baf54: serviceaccount "payload-processing-e2e-worker-w1-1baf54" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w1-1baf54

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-1baf54-666476dc6b

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

openshift-ingress

multus

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.989s (2.989s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.17s (3.17s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-9f7183-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-9f7183-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-b2eadb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-b2eadb-7d764cd74 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-b2eadb-7d764cd74

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-b2eadb-7d764cd74

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-b2eadb-7d764cd74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-b2eadb: serviceaccount "payload-processing-e2e-worker-w4-b2eadb" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-b2eadb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-b2eadb: serviceaccount "payload-processing-e2e-worker-w4-b2eadb" not found

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

Unhealthy

Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

ProbeError

Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

ProbeError

Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

Unhealthy

Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-9f7183-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-9f7183-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-00cfe6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-00cfe6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

ProbeError

Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-1baf54-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-1baf54-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w3-9f7183-openshift-default-646bd94655

SuccessfulCreate

Created pod: e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulCreate

Created pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

Started

Container started

openshift-ingress

multus

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

Unhealthy

Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

ProbeError

Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 1 to 2

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 5 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-7w5l8

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target
(x5)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

ProbeError

Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

Unhealthy

Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-wbbff

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 4 to 3

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-wbbff

openshift-ingress

replicaset-controller

e2e-worker-w2-00cfe6-openshift-default-64579789c7

SuccessfulCreate

Created pod: e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

openshift-ingress

deployment-controller

e2e-worker-w2-00cfe6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-00cfe6-openshift-default-64579789c7 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-00cfe6-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

Started

Container started

opendatahub

kubelet

test-curl-7633-346153

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7633-346153

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7633-346153

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.313s (3.313s including waiting). Image size: 106100986 bytes.
(x6)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused

opendatahub

kubelet

test-curl-7633-346153

Started

Container started

opendatahub

kubelet

test-curl-7633-346153

Created

Container created

opendatahub

kubelet

test-curl-7633-346153

Killing

Stopping container test-curl-7633-346153

opendatahub

multus

test-curl-7633-ac7032

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7633-ac7032

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7633-ac7032

Started

Container started

opendatahub

kubelet

test-curl-7633-ac7032

Created

Container created

opendatahub

kubelet

test-curl-7633-ac7032

Killing

Stopping container test-curl-7633-ac7032

openshift-ingress

replicaset-controller

e2e-extmod-a-5520b3-openshift-default-6848bdfc

SuccessfulCreate

Created pod: e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-5520b3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-5520b3

openshift-ingress

deployment-controller

e2e-extmod-a-5520b3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-5520b3-openshift-default-6848bdfc from 0 to 1

openshift-ingress

multus

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-5520b3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-5520b3-78b5854954 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-5520b3-78b5854954

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-5520b3-78b5854954-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-5520b3: serviceaccount "payload-processing-e2e-extmod-a-5520b3" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-5520b3-78b5854954

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-5520b3: serviceaccount "payload-processing-e2e-extmod-a-5520b3" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-5520b3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-a96a99-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-a96a99-openshift-default-6ccb446cf5 from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulCreate

Created pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

Created

Container created

openshift-ingress

multus

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 2 to 3

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 3 to 2

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-jkdvh

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

ProbeError

Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

Unhealthy

Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

Unhealthy

Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

ProbeError

Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

Created

Container created
(x4)

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

Unhealthy

Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

ProbeError

Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5520b3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5520b3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w2-00cfe6-openshift-default-64579789c7

SuccessfulCreate

Created pod: e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

openshift-ingress

multus

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-00cfe6-openshift-default-64579789c7

SuccessfulCreate

Created pod: e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-00cfe6-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w2-00cfe6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-00cfe6-openshift-default-64579789c7 from 2 to 4

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

Created

Container created

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-cae0ab

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-cae0ab

openshift-ingress

replicaset-controller

e2e-extmod-b-cae0ab-openshift-default-b65d947d9

SuccessfulCreate

Created pod: e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

openshift-ingress

deployment-controller

e2e-extmod-b-cae0ab-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-cae0ab-openshift-default-b65d947d9 from 0 to 1

openshift-ingress

multus

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

Unhealthy

Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

ProbeError

Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5520b3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-5520b3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-cae0ab

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-cae0ab-66775db5db from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-cae0ab-66775db5db

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-cae0ab-66775db5db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-cae0ab: serviceaccount "payload-processing-e2e-extmod-b-cae0ab" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-cae0ab

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-cae0ab-66775db5db

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-cae0ab: serviceaccount "payload-processing-e2e-extmod-b-cae0ab" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-m7jld

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-m7jld

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 2 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-6db2a7-openshift-default-77779b578f

SuccessfulCreate

Created pod: e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-6db2a7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-6db2a7

openshift-ingress

deployment-controller

e2e-extmod-a-6db2a7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-6db2a7-openshift-default-77779b578f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-6db2a7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-6db2a7-549f767475 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-6db2a7-5884755cfb

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-6db2a7-5884755cfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-6db2a7: serviceaccount "payload-processing-e2e-extmod-a-6db2a7" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-6db2a7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-6db2a7-5884755cfb from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-6db2a7: serviceaccount "payload-processing-e2e-extmod-a-6db2a7" not found

openshift-ingress

multus

payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-6db2a7-5884755cfb

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-9f7183-openshift-default-646bd94655

SuccessfulDelete

Deleted pod: e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-9f7183-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w3-9f7183-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-9f7183-openshift-default-646bd94655 from 2 to 1

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-7fsjm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-b1047d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-b1047d

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 3 to 4

openshift-ingress

replicaset-controller

e2e-extmod-a-b1047d-openshift-default-58cd5899d5

SuccessfulCreate

Created pod: e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

openshift-ingress

deployment-controller

e2e-extmod-a-b1047d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-b1047d-openshift-default-58cd5899d5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-b1047d: serviceaccount "payload-processing-e2e-extmod-a-b1047d" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-b1047d-68b886648

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-b1047d-68b886648-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-b1047d: serviceaccount "payload-processing-e2e-extmod-a-b1047d" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-b1047d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-b1047d-68b886648 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-b1047d-68b886648

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-b1047d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t

Started

Container started

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7b4675d6f7 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-7b4675d6f7

SuccessfulCreate

Created pod: payload-processing-external-model-7b4675d6f7-sq4wn
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-7b4675d6f7

FailedCreate

Error creating: pods "payload-processing-external-model-7b4675d6f7-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

kubelet

payload-processing-external-model-7b4675d6f7-sq4wn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-7b4675d6f7-sq4wn

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-7b4675d6f7-sq4wn

Created

Container created

models-as-a-service

multus

payload-processing-external-model-7b4675d6f7-sq4wn

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-6fddb87f65

SuccessfulCreate

Created pod: payload-processing-external-model-6fddb87f65-hxqhz

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulCreate

Created pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6fddb87f65 from 0 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w0-a96a99-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-a96a99-openshift-default-6ccb446cf5 from 3 to 5

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulCreate

Created pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

AddedInterface

Add eth0 [10.133.0.64/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-6fddb87f65-hxqhz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6fddb87f65-hxqhz

Created

Container created

models-as-a-service

multus

payload-processing-external-model-6fddb87f65-hxqhz

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6fddb87f65-hxqhz

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-cae0ab-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-cae0ab-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

Unhealthy

Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

ProbeError

Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

Unhealthy

Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

ProbeError

Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

ProbeError

Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

Unhealthy

Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused

models-as-a-service

replicaset-controller

payload-processing-external-model-7b4675d6f7

SuccessfulDelete

Deleted pod: payload-processing-external-model-7b4675d6f7-sq4wn

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-7b4675d6f7 from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-7b4675d6f7-sq4wn

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-dpdg9
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6db2a7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6db2a7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-98jdc

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-q6lf6
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 1 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

Started

Container started
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-cae0ab-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-cae0ab-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

Unhealthy

Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

Unhealthy

Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

ProbeError

Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

ProbeError

Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

Unhealthy

Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

ProbeError

Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6db2a7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-6db2a7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

ProbeError

Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

Unhealthy

Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-extmod-b-d057d7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-d057d7-openshift-default-6654f4d8fc from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-d057d7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-d057d7

openshift-ingress

replicaset-controller

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc

SuccessfulCreate

Created pod: e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

openshift-ingress

multus

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-d057d7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-d057d7-687bd5c98d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-d057d7-687bd5c98d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-d057d7: serviceaccount "payload-processing-e2e-extmod-b-d057d7" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-d057d7: serviceaccount "payload-processing-e2e-extmod-b-d057d7" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-d057d7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-d057d7-687bd5c98d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-d057d7-687bd5c98d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r

Started

Container started
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-b1047d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-b1047d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-b1047d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-b1047d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-1225f3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-1225f3

openshift-ingress

replicaset-controller

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69

SuccessfulCreate

Created pod: e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

openshift-ingress

deployment-controller

e2e-extmod-b-1225f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69 from 0 to 1

openshift-ingress

multus

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 4 to 5

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

Started

Container started

models-as-a-service

replicaset-controller

payload-processing-external-model-5dddfc6875

SuccessfulCreate

Created pod: payload-processing-external-model-5dddfc6875-9n4rm

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5dddfc6875 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-5dddfc6875-9n4rm

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-5dddfc6875-9n4rm

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-5dddfc6875-9n4rm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-5dddfc6875-9n4rm

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

ProbeError

Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-fn6tt

Unhealthy

Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-1225f3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-1225f3-5b668b7888 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-1225f3-5b668b7888

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-1225f3-5b668b7888-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-1225f3: serviceaccount "payload-processing-e2e-extmod-b-1225f3" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-1225f3: serviceaccount "payload-processing-e2e-extmod-b-1225f3" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-1225f3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-1225f3-5b668b7888

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

AddedInterface

Add eth0 [10.132.0.87/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6

AddedInterface

Add eth0 [10.132.0.88/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 5 to 10

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

AddedInterface

Add eth0 [10.132.0.90/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-449zx

AddedInterface

Add eth0 [10.132.0.89/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

Killing

Stopping container payload-pre-processing
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

ProbeError

Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

ProbeError

Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

Unhealthy

Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

Unhealthy

Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

Unhealthy

Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

ProbeError

Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Unhealthy

Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

ProbeError

Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Unhealthy

Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

ProbeError

Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-1baf54-5dd6c8447f-k8fgx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-1baf54-666476dc6b-gj2pq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-5520b3-78b5854954-4pltn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-5520b3-599b6d8447-z2cnt

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-extmod-a-5520b3-openshift-default-6848bdfc-l5vpw

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-2jpkz

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

Started

Container started

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2

openshift-ingress

multus

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc

SuccessfulCreate

Created pod: e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

openshift-ingress

replicaset-controller

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc

SuccessfulCreate

Created pod: e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

openshift-ingress

deployment-controller

e2e-worker-w1-1baf54-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-1baf54-openshift-default-84dc6cddcc from 1 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-1baf54-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

multus

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-cae0ab-66775db5db-j4xst

Killing

Stopping container payload-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d057d7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d057d7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-cae0ab-7bc4798f9-7qbwq

Killing

Stopping container payload-pre-processing
(x6)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

ProbeError

Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

Unhealthy

Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

ProbeError

Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

Unhealthy

Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-2zg9n

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-gdl9h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d057d7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d057d7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

ProbeError

Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w1-1baf54-openshift-default-84dc6cddcc-f9978

Unhealthy

Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

Unhealthy

Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

ProbeError

Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-cae0ab-openshift-default-b65d947d9-tkfd6

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1225f3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1225f3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

Killing

Stopping container payload-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1225f3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1225f3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-9f7183-764969574c-pj9mt

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-9f7183-84b59f48b4-mpc5v

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-00cfe6-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w2-00cfe6-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-00cfe6-openshift-default-64579789c7 from 4 to 2

openshift-ingress

replicaset-controller

e2e-worker-w2-00cfe6-openshift-default-64579789c7

SuccessfulDelete

Deleted pod: e2e-worker-w2-00cfe6-openshift-default-64579789c7-sk6xd

openshift-ingress

replicaset-controller

e2e-worker-w2-00cfe6-openshift-default-64579789c7

SuccessfulDelete

Deleted pod: e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

openshift-ingress

multus

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

AddedInterface

Add eth0 [10.132.0.97/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-9f7183-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

deployment-controller

e2e-worker-w3-9f7183-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-9f7183-openshift-default-646bd94655 from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w3-9f7183-openshift-default-646bd94655

SuccessfulCreate

Created pod: e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

ProbeError

Readiness probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-gsxkw

Unhealthy

Readiness probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

ProbeError

Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

Unhealthy

Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-dvd74

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-9f7183-openshift-default-646bd94655-tgk4f

Killing

Stopping container istio-proxy
(x9)

models-as-a-service

kubelet

payload-processing-external-model-6fddb87f65-hxqhz

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-bec17d7cdfd4" not found

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 4 to 3

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulDelete

Deleted pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-nl22s

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulDelete

Deleted pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulDelete

Deleted pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w0-a96a99-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-a96a99-openshift-default-6ccb446cf5 from 5 to 3

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-rvhfs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-tmzjk

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulDelete

Deleted pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-ffnd9

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 3 to 2
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-6db2a7-549f767475-pltcv

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-6db2a7-5884755cfb-td2rm

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-5dddfc6875-9n4rm

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-7789777d6-l8vds

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-7789777d6-l8vds

AddedInterface

Add eth0 [10.132.0.100/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-7789777d6-l8vds

Created

Container created

models-as-a-service

replicaset-controller

payload-processing-external-model-5dddfc6875

SuccessfulDelete

Deleted pod: payload-processing-external-model-5dddfc6875-9n4rm

models-as-a-service

kubelet

payload-processing-external-model-7789777d6-l8vds

Started

Container started

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7789777d6 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-7789777d6

SuccessfulCreate

Created pod: payload-processing-external-model-7789777d6-l8vds

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-5dddfc6875 from 1 to 0
(x24)

models-as-a-service

kubelet

payload-processing-external-model-5dddfc6875-9n4rm

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-a-6db2a7-openshift-default-77779b578f-smk2s

Killing

Stopping container istio-proxy

models-as-a-service

kubelet

payload-processing-external-model-6fddb87f65-hxqhz

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-6fddb87f65 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-6fddb87f65

SuccessfulDelete

Deleted pod: payload-processing-external-model-6fddb87f65-hxqhz

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d057d7-687bd5c98d-n7m6r

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d057d7-7678c7db67-k5x82

Killing

Stopping container payload-pre-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-765779c7c6

SuccessfulCreate

Created pod: payload-processing-external-model-765779c7c6-wn22v

models-as-a-service

kubelet

payload-processing-external-model-765779c7c6-wn22v

Created

Container created

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-765779c7c6 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-765779c7c6-wn22v

Started

Container started

models-as-a-service

multus

payload-processing-external-model-765779c7c6-wn22v

AddedInterface

Add eth0 [10.132.0.102/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-765779c7c6-wn22v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-d057d7-openshift-default-6654f4d8fc-g6vwc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b2eadb-7d764cd74-9j4ld

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b2eadb-668f7d8bdc-vkhcp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 2 to 4

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

AddedInterface

Add eth0 [10.133.0.71/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

Created

Container created
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-00cfe6-6797d7fb67-sqgbg

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-00cfe6-84444cfdcd-chqm6

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-fbkvr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

ProbeError

Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

Unhealthy

Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

ProbeError

Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

Unhealthy

Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

ProbeError

Readiness probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w2-00cfe6-openshift-default-64579789c7-xh7f6

Unhealthy

Readiness probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Unhealthy

Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

Unhealthy

Readiness probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

ProbeError

Readiness probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

ProbeError

Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-worker-w4-b2eadb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc from 4 to 6

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6

AddedInterface

Add eth0 [10.132.0.107/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-b2eadb-openshift-default

SuccessfulRescale

New size: 6; reason:

openshift-ingress

replicaset-controller

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc

SuccessfulCreate

Created pod: e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6

Created

Container created

openshift-ingress

multus

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-hl5mt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-cwhb6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-xl9n6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-sdbwc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-q8tcv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-b2eadb-openshift-default-54d5bd64fc-2ftq2

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulDelete

Deleted pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

openshift-ingress

deployment-controller

e2e-worker-w0-a96a99-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-a96a99-openshift-default-6ccb446cf5 from 3 to 2

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-mbxzj

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-2jpkz

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-2jpkz

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target
(x10)

models-as-a-service

kubelet

payload-processing-external-model-7789777d6-l8vds

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-ca76f974b55b" not found
(x32)

models-as-a-service

kubelet

payload-processing-external-model-765779c7c6-wn22v

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

e2e-worker-w0-a96a99-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-a96a99-openshift-default-6ccb446cf5 from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulDelete

Deleted pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-w9v9z

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-5db77fbd9f from 10 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-449zx

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-98jdc

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-qch24

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-b7kv4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-qch24

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-gh8zv

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-2lkmv

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-449zx

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-dpdg9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-98jdc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-q6lf6

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

Unhealthy

Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-t9ljt

ProbeError

Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

models-as-a-service

kubelet

payload-processing-external-model-7789777d6-l8vds

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-765779c7c6-wn22v

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-7789777d6-l8vds

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : [secret "agc-cred-5e61507d7978" not found, secret "agc-cred-ca76f974b55b" not found]

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-b1047d-68b886648-jmz6t

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-b1047d-5b8bbd9947-kkkkc

Killing

Stopping container payload-pre-processing

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

AddedInterface

Add eth0 [10.132.0.110/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5db77fbd9f

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5db77fbd9f-78bvs

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5db77fbd9f from 1 to 3

openshift-ingress

multus

maas-default-gateway-openshift-default-5db77fbd9f-78bvs

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-78bvs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-78bvs

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-78bvs

Started

Container started

openshift-ingress

kubelet

e2e-extmod-a-b1047d-openshift-default-58cd5899d5-7n9xw

Killing

Stopping container istio-proxy
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-78bvs

Unhealthy

Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

Unhealthy

Startup probe failed: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-8xfgm

ProbeError

Startup probe error: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5db77fbd9f-78bvs

ProbeError

Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1225f3-5b668b7888-lr2d6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1225f3-8494f5bf47-grtcx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-1225f3-openshift-default-5d7dd5fb69-x5qkh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-a96a99-757ff68588-jfr94

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-a96a99-6576946687-h9wjk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

AddedInterface

Add eth0 [10.132.0.113/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-a96a99-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

e2e-worker-w0-a96a99-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-a96a99-openshift-default-6ccb446cf5 from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5

SuccessfulCreate

Created pod: e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

Unhealthy

Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

ProbeError

Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-sbp2p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-a96a99-openshift-default-6ccb446cf5-s6cqr

Killing

Stopping container istio-proxy