Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd to ip-10-0-142-191.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-dab533

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-dab533: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-055a14

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-055a14: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-3d082d

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-3d082d: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-3f76d0

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-3f76d0: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-24bea9

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-24bea9: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w1-90ad0f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w1-90ad0f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-29e7e8

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-29e7e8: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-3bf869

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-3bf869: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-196e7e

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-196e7e: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-8dffj

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-8dffj to ip-10-0-142-191.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-6ccf9f4fbd-mg95w

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6ccf9f4fbd-mg95w to ip-10-0-142-191.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-6945d56d5f-5trg4

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6945d56d5f-5trg4 to ip-10-0-142-191.us-west-1.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-c7flb

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-c7flb to ip-10-0-134-204.us-west-1.compute.internal

opendatahub

ai-gateway-controller-5d8dddb676-tvx2w

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-tvx2w to ip-10-0-134-204.us-west-1.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-b77xj

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-b77xj to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk to ip-10-0-142-191.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-x75p2 to ip-10-0-142-191.us-west-1.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-r9wmv

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv to ip-10-0-134-176.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-68df898747-shn7h

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-68df898747-shn7h to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql to ip-10-0-134-204.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-65c44bf774-5666d

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-65c44bf774-5666d to ip-10-0-142-191.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-5f69dbdb4d-vvrwp

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5f69dbdb4d-vvrwp to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8 to ip-10-0-142-191.us-west-1.compute.internal

opendatahub

ai-gateway-controller-7656484495-trb67

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-7656484495-trb67 to ip-10-0-134-204.us-west-1.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-xcpfl

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl to ip-10-0-134-176.us-west-1.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-kx5jb

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-kx5jb to ip-10-0-142-191.us-west-1.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-jtkx7

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-jtkx7 to ip-10-0-142-191.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-gkgm9

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9 to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-7hkvk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-7hkvk to ip-10-0-134-204.us-west-1.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl to ip-10-0-142-191.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 to ip-10-0-134-176.us-west-1.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-9cbzm

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-9cbzm to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-77fbbd9bf8-td596

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-77fbbd9bf8-td596 to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-77fbbd9bf8-5zczj

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj to ip-10-0-134-176.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l to ip-10-0-134-204.us-west-1.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xb2fb to ip-10-0-142-191.us-west-1.compute.internal

kuadrant-system

authorino-56c6c9ddd5-7sgdp

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-7sgdp to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26 to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9 to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh to ip-10-0-142-191.us-west-1.compute.internal

kuadrant-system

authorino-57f7f8659f-n26k6

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-n26k6 to ip-10-0-142-191.us-west-1.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-htz9j

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-htz9j to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6 to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d to ip-10-0-134-204.us-west-1.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk to ip-10-0-142-191.us-west-1.compute.internal

kuadrant-system

authorino-69f65d44c7-8lctn

Scheduled

Successfully assigned kuadrant-system/authorino-69f65d44c7-8lctn to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2 to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-processing-7bb9489676-ckdnq

Scheduled

Successfully assigned openshift-ingress/payload-processing-7bb9489676-ckdnq to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j to ip-10-0-142-191.us-west-1.compute.internal

kuadrant-system

authorino-6f6944bcdd-pt749

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-pt749 to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44 to ip-10-0-134-176.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-0508d1

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-0508d1: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

router-default-56bdbb678c-mvkmq

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-56bdbb678c-mvkmq

FailedScheduling

no nodes available to schedule pods

openshift-ingress

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l to ip-10-0-134-204.us-west-1.compute.internal

kuadrant-system

authorino-86c994f77b-qwzgv

Scheduled

Successfully assigned kuadrant-system/authorino-86c994f77b-qwzgv to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8h87v to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

router-default-56bdbb678c-mvkmq

Scheduled

Successfully assigned openshift-ingress/router-default-56bdbb678c-mvkmq to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24 to ip-10-0-142-191.us-west-1.compute.internal

kuadrant-system

authorino-bddb76fbc-vtbff

Scheduled

Successfully assigned kuadrant-system/authorino-bddb76fbc-vtbff to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4 to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-8pm7p

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8pm7p to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-8rt47

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-b9chv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b9chv to ip-10-0-134-204.us-west-1.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-gdhhm

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-gdhhm to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd to ip-10-0-134-204.us-west-1.compute.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp to ip-10-0-134-204.us-west-1.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-tx25t

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-tx25t to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-c554b664c-9n829

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-c554b664c-9n829 to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7 to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh to ip-10-0-142-191.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-q9lfs

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs to ip-10-0-134-176.us-west-1.compute.internal

opendatahub

maas-controller-575cb5b89b-xd5wg

Scheduled

Successfully assigned opendatahub/maas-controller-575cb5b89b-xd5wg to ip-10-0-142-191.us-west-1.compute.internal

opendatahub

maas-controller-594b9656dc-jtm5p

Scheduled

Successfully assigned opendatahub/maas-controller-594b9656dc-jtm5p to ip-10-0-142-191.us-west-1.compute.internal

opendatahub

model-serving-api-75bd89fdfc-jvj68

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-jvj68 to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6kjfp to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9 to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-tnrml

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-tnrml to ip-10-0-142-191.us-west-1.compute.internal

opendatahub

test-curl-7348-b78501

Scheduled

Successfully assigned opendatahub/test-curl-7348-b78501 to ip-10-0-142-191.us-west-1.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-ckrpb

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-ckrpb to ip-10-0-134-176.us-west-1.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-67t5h

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-67t5h to ip-10-0-142-191.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-a8ef09

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-a8ef09: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll to ip-10-0-142-191.us-west-1.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-k7wrt

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-k7wrt to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-vrvfp

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-vrvfp to ip-10-0-134-204.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-g65d7

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-g65d7 to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p to ip-10-0-134-176.us-west-1.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-nfhp7

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7 to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv to ip-10-0-134-204.us-west-1.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5 to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x to ip-10-0-134-204.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r927w to ip-10-0-134-204.us-west-1.compute.internal

opendatahub

odh-model-controller-769875db56-l8pbh

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-l8pbh to ip-10-0-134-204.us-west-1.compute.internal

opendatahub

test-curl-7348-17efe5

Scheduled

Successfully assigned opendatahub/test-curl-7348-17efe5 to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj to ip-10-0-134-176.us-west-1.compute.internal

openshift-jobset-operator

jobset-controller-manager-f46d769d6-8vbck

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw to ip-10-0-134-204.us-west-1.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-hf7tr

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr to ip-10-0-134-176.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5 to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-r74jj to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f to ip-10-0-142-191.us-west-1.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-56bdbb678c from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-56bdbb678c

FailedCreate

Error creating: pods "router-default-56bdbb678c-" is forbidden: error fetching namespace "openshift-ingress": unable to find annotation openshift.io/sa.scc.uid-range

openshift-ingress

replicaset-controller

router-default-56bdbb678c

SuccessfulCreate

Created pod: router-default-56bdbb678c-mvkmq
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-56bdbb678c-mvkmq

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-56bdbb678c-mvkmq

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-56bdbb678c-mvkmq

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-56bdbb678c-mvkmq

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-56bdbb678c-mvkmq

Started

Container started

openshift-ingress

multus

router-default-56bdbb678c-mvkmq

AddedInterface

Add eth0 [10.132.0.20/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-56bdbb678c-mvkmq

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

router-default-56bdbb678c-mvkmq

Created

Container created

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-vrvfp

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-vrvfp

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-vrvfp

AddedInterface

Add eth0 [10.132.0.26/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-vrvfp

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.698s (4.698s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-vrvfp

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-vrvfp

Created

Container created

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-c554b664c

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-c554b664c-9n829

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-c554b664c from 0 to 1

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-c554b664c-9n829

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-c554b664c-9n829

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-c554b664c-9n829

Started

Container started

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-c554b664c-9n829

Created

Container created

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-c554b664c-9n829

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 14.323s (14.323s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 14.684s (14.684s including waiting). Image size: 183555333 bytes.
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-7f9fcc96f6

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-7f9fcc96f6

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-7f9fcc96f6 from 0 to 3

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-7f9fcc96f6

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

AddedInterface

Add eth0 [10.133.0.15/23] from ovn-kubernetes

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x3)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.335s (4.335s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.212s (4.212s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 5.295s (5.295s including waiting). Image size: 273709574 bytes.
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

NeedsReinstall

calculated deployment install is bad

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 782ms (782ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Started

Container started
(x2)

opendatahub-operator-system

operator-lifecycle-manager

install-dlnrn

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-x75p2

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 797ms (797ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 863ms (863ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Started

Container started

opendatahub-operator-system

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l_57195e19-fd43-413f-bc20-717d4052da19

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-7f9fcc96f6-b4g4l_57195e19-fd43-413f-bc20-717d4052da19 became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7f9fcc96f6-p2xld

Created

Container created
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-kx5jb

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-kx5jb

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-kx5jb

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.
(x6)

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

FailedCreate

Error creating: pods "cert-manager-webhook-5dbbbf75cc-" is forbidden: error looking up service account cert-manager/cert-manager-webhook: serviceaccount "cert-manager-webhook" not found

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-xcpfl

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-xcpfl

AddedInterface

Add eth0 [10.133.0.16/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-xcpfl

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found
(x11)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-xcpfl

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-kx5jb

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-kx5jb

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-kx5jb

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 6.078s (6.078s including waiting). Image size: 224874686 bytes.

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-xcpfl

Created

Container created

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-xcpfl

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 4.59s (4.59s including waiting). Image size: 309650901 bytes.

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-kx5jb_42d57e4e-d9ab-4ef9-bcf2-f9186e864dbe became leader

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-r9wmv

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-r9wmv

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-r9wmv

Created

Container created

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-r9wmv

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-r9wmv

AddedInterface

Add eth0 [10.133.0.17/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-htz9j

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-htz9j

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-htz9j

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-8dffj

cert-manager

multus

cert-manager-597b95cb84-8dffj

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-8dffj

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-tx25t

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-htz9j

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-htz9j

Started

Container started

kuadrant-system

multus

authorino-operator-7b7bb4bc77-tx25t

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-htz9j

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 4.549s (4.549s including waiting). Image size: 177967631 bytes.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

limitador-operator-controller-manager-5778bb4449-htz9j_0e184aa0-76fa-4e84-9a06-cae60292b0b8

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-htz9j_0e184aa0-76fa-4e84-9a06-cae60292b0b8 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-tx25t

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

cert-manager

kubelet

cert-manager-597b95cb84-8dffj

Created

Container created

cert-manager

kubelet

cert-manager-597b95cb84-8dffj

Started

Container started

cert-manager

kubelet

cert-manager-597b95cb84-8dffj

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 4.54s (4.54s including waiting). Image size: 309650901 bytes.

kuadrant-system

authorino-operator-7b7bb4bc77-tx25t_fd19543f-8592-4637-a889-611d201e0dfd

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-tx25t_fd19543f-8592-4637-a889-611d201e0dfd became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-tx25t

Started

Container started

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-tx25t

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 3.303s (3.303s including waiting). Image size: 167738465 bytes.

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-tx25t

Created

Container created

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-g65d7

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-g65d7

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-g65d7

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-nfhp7

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-nfhp7

AddedInterface

Add eth0 [10.133.0.18/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-nfhp7

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-g65d7

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 9.782s (9.782s including waiting). Image size: 424244516 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-g65d7

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-g65d7

Created

Container created

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-g65d7_f8a95e4b-47cd-4c64-a30c-4b0bf6ab1bb8

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-g65d7_f8a95e4b-47cd-4c64-a30c-4b0bf6ab1bb8 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-nfhp7

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 8.808s (8.808s including waiting). Image size: 190804149 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-nfhp7

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-nfhp7

Started

Container started

kuadrant-system

dns-operator-controller-manager-8557cbdb84-nfhp7_f9715d55-226a-4d54-a419-15f282c1450c

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-nfhp7_f9715d55-226a-4d54-a419-15f282c1450c became leader

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-hf7tr

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-hf7tr

AddedInterface

Add eth0 [10.133.0.19/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-hf7tr

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-fz6x8"

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-ckrpb

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-6rzzf"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-ingress

replicaset-controller

kube-auth-proxy-77fbbd9bf8

SuccessfulCreate

Created pod: kube-auth-proxy-77fbbd9bf8-td596

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-77fbbd9bf8 from 0 to 2

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-ingress

replicaset-controller

kube-auth-proxy-77fbbd9bf8

SuccessfulCreate

Created pod: kube-auth-proxy-77fbbd9bf8-5zczj

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

kuadrant-system

multus

limitador-limitador-74dfcfb894-ckrpb

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ckrpb

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-ingress

multus

kube-auth-proxy-77fbbd9bf8-5zczj

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-5zczj

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-5f555b57d9

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk

Created

Container created

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-5f555b57d9pfjdk

Started

Container started

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-5f555b57d9 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-n26k6

openshift-ingress

multus

kube-auth-proxy-77fbbd9bf8-td596

AddedInterface

Add eth0 [10.132.0.28/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-td596

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

kuadrant-system

multus

authorino-57f7f8659f-n26k6

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

kuadrant-system

kubelet

authorino-57f7f8659f-n26k6

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-f46d769d6

SuccessfulCreate

Created pod: jobset-controller-manager-f46d769d6-8vbck

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-f46d769d6 from 0 to 1

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-gdhhm

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-7sgdp

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-td596

Created

Container created

openshift-jobset-operator

multus

jobset-controller-manager-f46d769d6-8vbck

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-controller-manager-f46d769d6-8vbck

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-5zczj

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-5zczj

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-td596

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.899s (2.899s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-td596

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-77fbbd9bf8-5zczj

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.947s (2.947s including waiting). Image size: 188904677 bytes.

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-gdhhm

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

kubelet

authorino-56c6c9ddd5-7sgdp

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

multus

limitador-limitador-7c9b87d574-gdhhm

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes

kuadrant-system

multus

authorino-56c6c9ddd5-7sgdp

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-57f7f8659f-n26k6

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-hf7tr

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-hf7tr

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 7.653s (7.653s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-hf7tr

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-n26k6

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.762s (3.762s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-57f7f8659f-n26k6

Started

Container started

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

kubelet

jobset-controller-manager-f46d769d6-8vbck

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 4.221s (4.221s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-f46d769d6-8vbck

Started

Container started

openshift-jobset-operator

jobset-controller-manager-f46d769d6-8vbck_6367cae1-605f-4a88-a71a-44bf5a5f173d

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-f46d769d6-8vbck_6367cae1-605f-4a88-a71a-44bf5a5f173d became leader

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-gdhhm

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 3.956s (3.956s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-gdhhm

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-n26k6

Killing

Stopping container authorino

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-gdhhm

Created

Container created

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-n26k6

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

authorino-56c6c9ddd5-7sgdp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.954s (3.954s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-7sgdp

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-f46d769d6-8vbck

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-7sgdp

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ckrpb

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ckrpb

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 14.698s (14.698s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ckrpb

Created

Container created

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-ckrpb

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ckrpb

Killing

Stopping container limitador

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-jtkx7

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-jtkx7

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-jtkx7

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-jtkx7

Started

Container started

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-jtkx7

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-jtkx7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 3.843s (3.843s including waiting). Image size: 195729000 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kserve-module-controller-manager-7566cf5fc8-jtkx7_1fae9b02-4fc2-4d03-994e-176335284109

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-jtkx7_1fae9b02-4fc2-4d03-994e-176335284109 became leader

opendatahub

replicaset-controller

maas-controller-594b9656dc

SuccessfulCreate

Created pod: maas-controller-594b9656dc-jtm5p

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-594b9656dc from 0 to 1

opendatahub

replicaset-controller

maas-controller-575cb5b89b

SuccessfulCreate

Created pod: maas-controller-575cb5b89b-xd5wg

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-575cb5b89b from 0 to 1

opendatahub

multus

maas-controller-594b9656dc-jtm5p

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-594b9656dc-jtm5p

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:25a1c9f74f9ea6b313a03f7c231930d08023e49ea515da9c170cb81c23a8195f"

opendatahub

multus

maas-controller-575cb5b89b-xd5wg

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-575cb5b89b-xd5wg

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:25a1c9f74f9ea6b313a03f7c231930d08023e49ea515da9c170cb81c23a8195f" in 8.502s (8.502s including waiting). Image size: 200103413 bytes.

opendatahub

kubelet

maas-controller-594b9656dc-jtm5p

Started

Container started

opendatahub

kubelet

maas-controller-594b9656dc-jtm5p

Created

Container created

opendatahub

kubelet

maas-controller-594b9656dc-jtm5p

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:25a1c9f74f9ea6b313a03f7c231930d08023e49ea515da9c170cb81c23a8195f" in 8.843s (8.843s including waiting). Image size: 200103413 bytes.

opendatahub

maas-controller-594b9656dc-jtm5p_04dace29-bcaf-4ead-9cde-91f0844f5798

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-594b9656dc-jtm5p_04dace29-bcaf-4ead-9cde-91f0844f5798 became leader

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

FailedMount

MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

FailedMount

MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-594b9656dc from 1 to 0

opendatahub

replicaset-controller

maas-controller-594b9656dc

SuccessfulDelete

Deleted pod: maas-controller-594b9656dc-jtm5p

opendatahub

kubelet

maas-controller-594b9656dc-jtm5p

Killing

Stopping container manager

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 16.985s (16.985s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xb2fb

Created

Container created

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-8pm7p

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8pm7p

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8pm7p

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8pm7p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-8pm7p

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8pm7p

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8pm7p

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

opendatahub

maas-controller-575cb5b89b-xd5wg_629ceca6-38da-4c5a-9906-e776ad136e59

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-575cb5b89b-xd5wg_629ceca6-38da-4c5a-9906-e776ad136e59 became leader

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-vrvfp

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-vrvfp

Killing

Stopping container discovery

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-q9lfs

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-q9lfs

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-l8pbh

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-jvj68

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

multus

kserve-controller-manager-776cc66d47-67t5h

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-67t5h

opendatahub

kubelet

kserve-controller-manager-776cc66d47-67t5h

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

kubelet

model-serving-api-75bd89fdfc-jvj68

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

multus

model-serving-api-75bd89fdfc-jvj68

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

opendatahub

kubelet

odh-model-controller-769875db56-l8pbh

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

multus

odh-model-controller-769875db56-l8pbh

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

opendatahub

kubelet

odh-model-controller-769875db56-l8pbh

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.293s (3.293s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-l8pbh

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-l8pbh

Started

Container started

opendatahub

odh-model-controller-769875db56-l8pbh_20e83386-946b-43b4-ab1c-0112423a9e8e

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-l8pbh_20e83386-946b-43b4-ab1c-0112423a9e8e became leader

opendatahub

kubelet

model-serving-api-75bd89fdfc-jvj68

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-jvj68

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 5.407s (5.407s including waiting). Image size: 184925761 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-jvj68

Started

Container started

opendatahub

kubelet

kserve-controller-manager-776cc66d47-67t5h

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 6.48s (6.48s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-67t5h

Started

Container started

opendatahub

kubelet

kserve-controller-manager-776cc66d47-67t5h

Created

Container created

opendatahub

kserve-controller-manager-776cc66d47-67t5h_40342405-5c3b-432f-a738-ab9aebfb71cf

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-67t5h_40342405-5c3b-432f-a738-ab9aebfb71cf became leader

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-q9lfs

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 6.95s (6.95s including waiting). Image size: 235100981 bytes.
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-q9lfs

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1
(x2)

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-q9lfs

Created

Container created
(x2)

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-q9lfs

Started

Container started

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-pt749

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-q9lfs

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 792ms (792ms including waiting). Image size: 235100981 bytes.

kuadrant-system

replicaset-controller

authorino-bddb76fbc

SuccessfulCreate

Created pod: authorino-bddb76fbc-vtbff

kuadrant-system

multus

authorino-6f6944bcdd-pt749

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

kuadrant-system

multus

authorino-69f65d44c7-8lctn

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-69f65d44c7 from 1 to 0

kuadrant-system

replicaset-controller

authorino-69f65d44c7

SuccessfulCreate

Created pod: authorino-69f65d44c7-8lctn

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-bddb76fbc from 0 to 1

kuadrant-system

replicaset-controller

authorino-69f65d44c7

SuccessfulDelete

Deleted pod: authorino-69f65d44c7-8lctn

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-pt749

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-69f65d44c7 from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

Started

Container started
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-pt749

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

opendatahub

llmisvc-controller-manager-869d5dcc5d-q9lfs_cedf3f2d-09f1-4191-bf29-1f8b70ffac24

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-q9lfs_cedf3f2d-09f1-4191-bf29-1f8b70ffac24 became leader

kuadrant-system

kubelet

authorino-bddb76fbc-vtbff

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-bddb76fbc-vtbff

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-69f65d44c7-8lctn

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-6f6944bcdd-pt749

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.488s (1.488s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-69f65d44c7-8lctn

Started

Container started

kuadrant-system

kubelet

authorino-69f65d44c7-8lctn

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-pt749

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-6f6944bcdd-pt749

Created

Container created

kuadrant-system

kubelet

authorino-bddb76fbc-vtbff

Created

Container created

kuadrant-system

kubelet

authorino-69f65d44c7-8lctn

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.566s (1.566s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-bddb76fbc-vtbff

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.139s (1.139s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6f6944bcdd-pt749

Started

Container started

kuadrant-system

kubelet

authorino-bddb76fbc-vtbff

Started

Container started

kuadrant-system

kubelet

authorino-69f65d44c7-8lctn

Killing

Stopping container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-7sgdp

kuadrant-system

kubelet

authorino-56c6c9ddd5-7sgdp

Killing

Stopping container authorino
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

Unhealthy

Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r74jj

ProbeError

Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-gkgm9

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gkgm9

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-6466c554d8-gkgm9

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gkgm9

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gkgm9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 757ms (757ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gkgm9

Created

Container created
(x2)

opendatahub

kubelet

maas-controller-575cb5b89b-xd5wg

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:25a1c9f74f9ea6b313a03f7c231930d08023e49ea515da9c170cb81c23a8195f"
(x2)

opendatahub

kubelet

maas-controller-575cb5b89b-xd5wg

Started

Container started
(x2)

opendatahub

kubelet

maas-controller-575cb5b89b-xd5wg

Created

Container created

opendatahub

kubelet

maas-controller-575cb5b89b-xd5wg

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:25a1c9f74f9ea6b313a03f7c231930d08023e49ea515da9c170cb81c23a8195f" in 930ms (930ms including waiting). Image size: 200103413 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-q9lfs

Killing

Stopping container manager

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-q9lfs

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

maas-controller-575cb5b89b-xd5wg_3f55a6e0-bff1-49ab-b09e-b90e609b02d5

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-575cb5b89b-xd5wg_3f55a6e0-bff1-49ab-b09e-b90e609b02d5 became leader

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

multus

ai-gateway-controller-5d8dddb676-tvx2w

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-tvx2w

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tvx2w

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tvx2w

Created

Container created

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tvx2w

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 2.342s (2.342s including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tvx2w

Started

Container started

opendatahub

ai-gateway-controller-5d8dddb676-tvx2w_e4f01ec1-fb1c-46e6-9658-445c5847be8b

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-tvx2w_e4f01ec1-fb1c-46e6-9658-445c5847be8b became leader

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-7bb9489676 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-7hkvk

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

SuccessfulCreate

Created pod: payload-processing-7bb9489676-ckdnq
(x7)

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

FailedCreate

Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-pre-processing-7b98bb678c-7hkvk

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-7bb9489676-ckdnq

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-7bb9489676-ckdnq

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7hkvk

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

opendatahub

llmisvc-controller-manager-6466c554d8-gkgm9_58b8aae0-491f-4697-af97-ad508e918ed3

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-gkgm9_58b8aae0-491f-4697-af97-ad508e918ed3 became leader

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7hkvk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.192s (3.192s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-7bb9489676-ckdnq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.968s (2.968s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-7bb9489676-ckdnq

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7hkvk

Started

Container started

openshift-ingress

kubelet

payload-processing-7bb9489676-ckdnq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7hkvk

Created

Container created

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-7656484495 from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-7656484495

SuccessfulCreate

Created pod: ai-gateway-controller-7656484495-trb67

opendatahub

multus

ai-gateway-controller-7656484495-trb67

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-7656484495-trb67

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-7656484495-trb67

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 759ms (759ms including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-7656484495-trb67

Created

Container created

opendatahub

kubelet

ai-gateway-controller-7656484495-trb67

Started

Container started

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-tvx2w

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-tvx2w

opendatahub

ai-gateway-controller-7656484495-trb67_a195168a-61c4-4229-9a1b-31c1ca184276

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-7656484495-trb67_a195168a-61c4-4229-9a1b-31c1ca184276 became leader

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl

Created

Container created

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vj6cl

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.181s (2.181s including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-c7flb

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-c7flb

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-c7flb

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-c7flb

Started

Container started

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-c7flb

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.488s (1.488s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-c7flb

Created

Container created

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-b77xj

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-b77xj

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-b77xj

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-b77xj

Started

Container started

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-b77xj

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 580ms (580ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-b77xj

Created

Container created

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-9cbzm

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-9cbzm

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-9cbzm

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-9cbzm

Started

Container started

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-9cbzm

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 595ms (595ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-9cbzm

Created

Container created

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 585ms (585ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8fh5vl

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-k7wrt

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

kubelet

echo-simulator-kserve-64b6f8f96c-k7wrt

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

multus

echo-simulator-kserve-64b6f8f96c-k7wrt

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 1 to 2

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

llm

kubelet

echo-simulator-kserve-64b6f8f96c-k7wrt

Created

Container created

llm

kubelet

echo-simulator-kserve-64b6f8f96c-k7wrt

Started

Container started

llm

kubelet

echo-simulator-kserve-64b6f8f96c-k7wrt

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.527s (2.527s including waiting). Image size: 78208878 bytes.

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5

Created

Container created

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5

Started

Container started

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-zlps5

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 577ms (577ms including waiting). Image size: 98346788 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.063s (4.063s including waiting). Image size: 183555333 bytes.

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 640ms (640ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-qcrkp

Started

Container started
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

ProbeError

Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Unhealthy

Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Unhealthy

Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

ProbeError

Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.446s (3.446s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

Started

Container started

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

ProbeError

Startup probe error: Get "http://10.133.0.26:15021/healthz/ready": dial tcp 10.133.0.26:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf

Unhealthy

Startup probe failed: Get "http://10.133.0.26:15021/healthz/ready": dial tcp 10.133.0.26:15021: connect: connection refused

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-r927w

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 3 to 6

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

Started

Container started

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

Unhealthy

Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

ProbeError

Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused body:

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

Unhealthy

Startup probe failed: Get "http://10.133.0.27:15021/healthz/ready": dial tcp 10.133.0.27:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

ProbeError

Startup probe error: Get "http://10.133.0.27:15021/healthz/ready": dial tcp 10.133.0.27:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

ProbeError

Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

Unhealthy

Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused

kuadrant-system

replicaset-controller

authorino-86c994f77b

SuccessfulCreate

Created pod: authorino-86c994f77b-qwzgv

kuadrant-system

kubelet

authorino-86c994f77b-qwzgv

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-86c994f77b-qwzgv

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-bddb76fbc-vtbff

Killing

Stopping container authorino
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-bddb76fbc from 1 to 0

kuadrant-system

kubelet

authorino-86c994f77b-qwzgv

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.535s (1.535s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-86c994f77b-qwzgv

Created

Container created

kuadrant-system

kubelet

authorino-86c994f77b-qwzgv

Started

Container started

kuadrant-system

replicaset-controller

authorino-bddb76fbc

SuccessfulDelete

Deleted pod: authorino-bddb76fbc-vtbff

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-196e7e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-196e7e

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-29e7e8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-29e7e8

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-3bf869

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-3bf869

openshift-ingress

multus

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulCreate

Created pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

openshift-ingress

deployment-controller

e2e-worker-w2-29e7e8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-29e7e8-openshift-default-5d7f77986f from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w3-3bf869-openshift-default-556c65d7ff

SuccessfulCreate

Created pod: e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

openshift-ingress

deployment-controller

e2e-worker-w4-196e7e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-196e7e-openshift-default-5b9d665fbc from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w3-3bf869-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-3bf869-openshift-default-556c65d7ff from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc

SuccessfulCreate

Created pod: e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

openshift-ingress

multus

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-24bea9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-24bea9

openshift-ingress

multus

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulCreate

Created pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 0 to 1

openshift-ingress

multus

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-29e7e8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-29e7e8-db9db6774 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-90ad0f-openshift-default-588c897dc9

SuccessfulCreate

Created pod: e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

openshift-ingress

deployment-controller

e2e-worker-w1-90ad0f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-90ad0f-openshift-default-588c897dc9 from 0 to 1

openshift-ingress

multus

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-29e7e8: serviceaccount "payload-processing-e2e-worker-w2-29e7e8" not found

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w1-90ad0f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-90ad0f
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-29e7e8-db9db6774

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-29e7e8-db9db6774-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-29e7e8: serviceaccount "payload-processing-e2e-worker-w2-29e7e8" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-29e7e8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-29e7e8-db9db6774

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.015s (3.015s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.973s (2.973s including waiting). Image size: 125353963 bytes.

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-3bf869

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-3bf869-7f65466bb8 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-3bf869

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-3bf869-7f65466bb8

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-3bf869-7f65466bb8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-3bf869: serviceaccount "payload-processing-e2e-worker-w3-3bf869" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-3bf869-7f65466bb8

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-3bf869: serviceaccount "payload-processing-e2e-worker-w3-3bf869" not found

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h

Started

Container started
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-196e7e-699dcb9b76

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-196e7e-699dcb9b76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-196e7e: serviceaccount "payload-processing-e2e-worker-w4-196e7e" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-196e7e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-196e7e-7785d9947 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-196e7e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-196e7e-699dcb9b76 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-196e7e-699dcb9b76

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-196e7e: serviceaccount "payload-processing-e2e-worker-w4-196e7e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-24bea9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-24bea9-6d9576f896 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-24bea9-6d9576f896

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-24bea9-6d9576f896-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-24bea9: serviceaccount "payload-processing-e2e-worker-w0-24bea9" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-24bea9-6d9576f896

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-24bea9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-24bea9-697477cb47 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-24bea9: serviceaccount "payload-processing-e2e-worker-w0-24bea9" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w1-90ad0f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-90ad0f: serviceaccount "payload-processing-e2e-worker-w1-90ad0f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w1-90ad0f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-90ad0f: serviceaccount "payload-processing-e2e-worker-w1-90ad0f" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-3bf869-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-3bf869-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

ProbeError

Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

Unhealthy

Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

Unhealthy

Readiness probe failed: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

ProbeError

Readiness probe error: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90ad0f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90ad0f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

ProbeError

Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

Unhealthy

Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-196e7e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-196e7e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

Unhealthy

Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

ProbeError

Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

Created

Container created
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-3bf869-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-3bf869-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-196e7e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

ProbeError

Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

Unhealthy

Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-196e7e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90ad0f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90ad0f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulCreate

Created pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

openshift-ingress

multus

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 1 to 2

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

Unhealthy

Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

ProbeError

Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90ad0f-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w1-90ad0f-openshift-default-588c897dc9

SuccessfulCreate

Created pod: e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

openshift-ingress

deployment-controller

e2e-worker-w1-90ad0f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-90ad0f-openshift-default-588c897dc9 from 1 to 2

openshift-ingress

multus

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

ProbeError

Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

Unhealthy

Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-r927w

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-r927w

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7p6xf

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 6 to 4

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

Started

Container started

openshift-ingress

deployment-controller

e2e-worker-w2-29e7e8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-29e7e8-openshift-default-5d7f77986f from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulCreate

Created pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

ProbeError

Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

Unhealthy

Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused

opendatahub

kubelet

test-curl-7348-17efe5

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7348-17efe5

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7348-17efe5

Started

Container started

opendatahub

kubelet

test-curl-7348-17efe5

Created

Container created

opendatahub

kubelet

test-curl-7348-17efe5

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.451s (2.451s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7348-17efe5

Killing

Stopping container test-curl-7348-17efe5

opendatahub

kubelet

test-curl-7348-b78501

Created

Container created

opendatahub

multus

test-curl-7348-b78501

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7348-b78501

Started

Container started

opendatahub

kubelet

test-curl-7348-b78501

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7348-b78501

Killing

Stopping container test-curl-7348-b78501

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-a8ef09

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-a8ef09

openshift-ingress

deployment-controller

e2e-extmod-a-a8ef09-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-a8ef09-openshift-default-76cfc458b5 from 0 to 1

openshift-ingress

multus

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5

SuccessfulCreate

Created pod: e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-a8ef09: serviceaccount "payload-processing-e2e-extmod-a-a8ef09" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-a8ef09

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-a8ef09

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-a8ef09: serviceaccount "payload-processing-e2e-extmod-a-a8ef09" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w1-90ad0f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-90ad0f-openshift-default-588c897dc9 from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90ad0f-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w1-90ad0f-openshift-default-588c897dc9

SuccessfulCreate

Created pod: e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

openshift-ingress

multus

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

ProbeError

Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

Unhealthy

Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 4 to 3

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-jchrk

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7bdc57c4cf from 3 to 2

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7bdc57c4cf-8h87v

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulCreate

Created pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 2 to 3
(x2)

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

Started

Container started

openshift-ingress

multus

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

Unhealthy

Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

ProbeError

Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-extmod-b-055a14-openshift-default-7749c6d945

SuccessfulCreate

Created pod: e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-0508d1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-0508d1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-055a14

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-055a14

openshift-ingress

deployment-controller

e2e-extmod-a-0508d1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-0508d1-openshift-default-774969cd4f from 0 to 1

openshift-ingress

deployment-controller

e2e-extmod-b-055a14-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-055a14-openshift-default-7749c6d945 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-0508d1-openshift-default-774969cd4f

SuccessfulCreate

Created pod: e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

openshift-ingress

multus

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

multus

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulCreate

Created pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 3 to 4

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

Created

Container created

openshift-ingress

multus

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

Started

Container started

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-055a14

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-055a14-57564fbdb from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-055a14

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-055a14-57564fbdb

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-055a14-57564fbdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-055a14: serviceaccount "payload-processing-e2e-extmod-b-055a14" not found
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-055a14: serviceaccount "payload-processing-e2e-extmod-b-055a14" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-055a14-57564fbdb

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw

AddedInterface

Add eth0 [10.134.0.87/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a8ef09-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a8ef09-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-0508d1-5b945b46dd

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a8ef09-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a8ef09-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-0508d1: serviceaccount "payload-processing-e2e-extmod-a-0508d1" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-0508d1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-0508d1-5b945b46dd

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-0508d1-5b945b46dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-0508d1: serviceaccount "payload-processing-e2e-extmod-a-0508d1" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-0508d1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-0508d1-5b945b46dd from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk

AddedInterface

Add eth0 [10.134.0.88/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

AddedInterface

Add eth0 [10.134.0.89/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w1-90ad0f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-90ad0f-openshift-default-588c897dc9 from 3 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-90ad0f-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w1-90ad0f-openshift-default-588c897dc9

SuccessfulCreate

Created pod: e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

openshift-ingress

replicaset-controller

e2e-worker-w1-90ad0f-openshift-default-588c897dc9

SuccessfulCreate

Created pod: e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

Unhealthy

Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

ProbeError

Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

Created

Container created

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-dab533

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-dab533

openshift-ingress

multus

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-a-dab533-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-dab533-openshift-default-6668bd9897 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-dab533-openshift-default-6668bd9897

SuccessfulCreate

Created pod: e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26
(x5)

openshift-ingress

kubelet

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-dab533-7cb99f96

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-dab533-7cb99f96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-dab533: serviceaccount "payload-processing-e2e-extmod-a-dab533" not found

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulCreate

Created pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

openshift-ingress

deployment-controller

e2e-worker-w2-29e7e8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-29e7e8-openshift-default-5d7f77986f from 2 to 4

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-dab533

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-dab533-7cb99f96 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-dab533

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulCreate

Created pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-dab533: serviceaccount "payload-processing-e2e-extmod-a-dab533" not found

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc

SuccessfulCreate

Created pod: e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

Created

Container created

openshift-ingress

multus

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds

AddedInterface

Add eth0 [10.134.0.92/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

AddedInterface

Add eth0 [10.134.0.91/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w4-196e7e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-196e7e-openshift-default-5b9d665fbc from 1 to 3

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc

SuccessfulCreate

Created pod: e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh

AddedInterface

Add eth0 [10.134.0.93/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-055a14-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-055a14-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-dab533-7cb99f96

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh

openshift-ingress

multus

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-196e7e-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

AddedInterface

Add eth0 [10.134.0.94/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh

Started

Container started
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-65c44bf774

FailedCreate

Error creating: pods "payload-processing-external-model-65c44bf774-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-65c44bf774 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-65c44bf774

SuccessfulCreate

Created pod: payload-processing-external-model-65c44bf774-5666d

models-as-a-service

kubelet

payload-processing-external-model-65c44bf774-5666d

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-65c44bf774-5666d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-65c44bf774-5666d

AddedInterface

Add eth0 [10.134.0.95/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-65c44bf774-5666d

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-6945d56d5f-5trg4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

ProbeError

Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

Unhealthy

Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6945d56d5f from 0 to 1

models-as-a-service

multus

payload-processing-external-model-6945d56d5f-5trg4

AddedInterface

Add eth0 [10.134.0.96/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-6945d56d5f

SuccessfulCreate

Created pod: payload-processing-external-model-6945d56d5f-5trg4

models-as-a-service

kubelet

payload-processing-external-model-6945d56d5f-5trg4

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-6945d56d5f-5trg4

Created

Container created
(x7)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

Unhealthy

Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

Unhealthy

Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

ProbeError

Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

ProbeError

Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

ProbeError

Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

Unhealthy

Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 2 to 4

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-65c44bf774 from 1 to 0

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-65c44bf774-5666d

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-65c44bf774

SuccessfulDelete

Deleted pod: payload-processing-external-model-65c44bf774-5666d

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dab533-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dab533-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

ProbeError

Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

Unhealthy

Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-055a14-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-055a14-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-0508d1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-0508d1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-extmod-b-3f76d0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-3f76d0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-3f76d0

openshift-ingress

replicaset-controller

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc

SuccessfulCreate

Created pod: e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

openshift-ingress

multus

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-3f76d0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-3f76d0: serviceaccount "payload-processing-e2e-extmod-b-3f76d0" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-3f76d0: serviceaccount "payload-processing-e2e-extmod-b-3f76d0" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-3f76d0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv

AddedInterface

Add eth0 [10.134.0.99/23] from ovn-kubernetes
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-0508d1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-0508d1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3f76d0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3f76d0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6ccf9f4fbd from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-6ccf9f4fbd

SuccessfulCreate

Created pod: payload-processing-external-model-6ccf9f4fbd-mg95w

models-as-a-service

kubelet

payload-processing-external-model-6ccf9f4fbd-mg95w

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-6ccf9f4fbd-mg95w

AddedInterface

Add eth0 [10.134.0.102/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6ccf9f4fbd-mg95w

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-6ccf9f4fbd-mg95w

Created

Container created

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 4 to 5

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-90ad0f-75ccc5cb64-wqt24

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-90ad0f-cd6dcffff-46pcd

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

Created

Container created
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

Unhealthy

Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

ProbeError

Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

Unhealthy

Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

ProbeError

Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused body:
(x5)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7bdc57c4cf

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-7bdc57c4cf from 5 to 10

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-8rt47

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-tnrml

AddedInterface

Add eth0 [10.134.0.104/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5

AddedInterface

Add eth0 [10.134.0.105/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8rt47

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-tnrml

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-tnrml

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-tnrml

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8rt47

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8rt47

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7bdc57c4cf-b9chv

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-b9chv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-b9chv

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-b9chv

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8rt47

Unhealthy

Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-8rt47

ProbeError

Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused body:
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5

ProbeError

Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused body:
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-7dz5p

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-b9chv

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-tnrml

Unhealthy

Startup probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-tnrml

ProbeError

Startup probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-b9chv

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-m9tc5

Unhealthy

Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-extmod-b-3d082d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-3d082d-openshift-default-5d8b85969 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-3d082d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-3d082d

openshift-ingress

replicaset-controller

e2e-extmod-b-3d082d-openshift-default-5d8b85969

SuccessfulCreate

Created pod: e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

openshift-ingress

multus

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-p4x6f

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-5rc2j

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-hxqlh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dab533-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dab533-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-3d082d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-3d082d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-3d082d-7cb999b99b from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-3d082d-7cb999b99b

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-3d082d-7cb999b99b

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-3d082d-7cb999b99b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-3d082d: serviceaccount "payload-processing-e2e-extmod-b-3d082d" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-3d082d: serviceaccount "payload-processing-e2e-extmod-b-3d082d" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

AddedInterface

Add eth0 [10.134.0.107/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll

AddedInterface

Add eth0 [10.134.0.108/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

Created

Container created

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-6kjfp

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

AddedInterface

Add eth0 [10.134.0.109/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.966s (4.966s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a8ef09-6dc6dc469d-zt4s6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a8ef09-5bf9595dcf-gdxll

Killing

Stopping container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

ProbeError

Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

Unhealthy

Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-a8ef09-openshift-default-76cfc458b5-p5kjp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-055a14-57564fbdb-75w6p

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-055a14-94c9dcdbc-s2zjw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-b-055a14-openshift-default-7749c6d945-bq46l

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3f76d0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3f76d0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-3bf869-7f65466bb8-phb7h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3d082d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3d082d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-3bf869-cc6fb6c67-5674j

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3d082d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x9)

models-as-a-service

kubelet

payload-processing-external-model-6945d56d5f-5trg4

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-26d257c38a93" not found
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3d082d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-worker-w3-3bf869-openshift-default-556c65d7ff-vzqj9

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulDelete

Deleted pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 4 to 3

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-rjdd2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-0508d1-openshift-default-774969cd4f-xnvql

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3f76d0-b46c7f5c5-b2crv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3f76d0-658f7fbdb6-hcslq

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-0508d1-685fc967b7-nbplp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-0508d1-5b945b46dd-6mwfk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulDelete

Deleted pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w2-29e7e8-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-29e7e8-openshift-default-5d7f77986f from 4 to 3

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-rzwdd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x25)

models-as-a-service

kubelet

payload-processing-external-model-6ccf9f4fbd-mg95w

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

replicaset-controller

payload-processing-external-model-5f69dbdb4d

SuccessfulCreate

Created pod: payload-processing-external-model-5f69dbdb4d-vvrwp

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-6ccf9f4fbd from 1 to 0

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5f69dbdb4d from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-6ccf9f4fbd-mg95w

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-6ccf9f4fbd

SuccessfulDelete

Deleted pod: payload-processing-external-model-6ccf9f4fbd-mg95w

models-as-a-service

kubelet

payload-processing-external-model-5f69dbdb4d-vvrwp

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-5f69dbdb4d-vvrwp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5f69dbdb4d-vvrwp

Created

Container created

models-as-a-service

multus

payload-processing-external-model-5f69dbdb4d-vvrwp

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-196e7e-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-196e7e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-196e7e-openshift-default-5b9d665fbc from 3 to 2

openshift-ingress

replicaset-controller

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc

SuccessfulDelete

Deleted pod: e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 3 to 2

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulDelete

Deleted pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj

models-as-a-service

kubelet

payload-processing-external-model-6945d56d5f-5trg4

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-6945d56d5f from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-6945d56d5f

SuccessfulDelete

Deleted pod: payload-processing-external-model-6945d56d5f-5trg4

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

ProbeError

Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-gv8fd

Unhealthy

Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

Unhealthy

Readiness probe failed: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

Unhealthy

Readiness probe failed: Get "http://10.133.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj

ProbeError

Readiness probe error: Get "http://10.133.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7bdc57c4cf-l8xdw

ProbeError

Readiness probe error: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulDelete

Deleted pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w2-29e7e8-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-29e7e8-openshift-default-5d7f77986f from 3 to 2

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-68df898747 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-68df898747

SuccessfulCreate

Created pod: payload-processing-external-model-68df898747-shn7h

models-as-a-service

kubelet

payload-processing-external-model-68df898747-shn7h

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-68df898747-shn7h

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-68df898747-shn7h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-68df898747-shn7h

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-24bea9-697477cb47-h7cfh

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-24bea9-6d9576f896-qhjkg

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulCreate

Created pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

SuccessfulCreate

Created pod: e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-24bea9-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-qp86d

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 0 to 1

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w0-24bea9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c from 2 to 4
(x3)

openshift-ingress

replicaset-controller

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c

FailedCreate

Error creating: pods "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-" is forbidden: error looking up service account openshift-ingress/e2e-worker-w0-24bea9-openshift-default: serviceaccount "e2e-worker-w0-24bea9-openshift-default" not found
(x2)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf

FailedMount

MountVolume.SetUp failed for volume "istio-token" : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
(x2)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-xhfzf

FailedMount

MountVolume.SetUp failed for volume "kube-api-access-hqc95" : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
(x2)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm

FailedMount

MountVolume.SetUp failed for volume "istio-token" : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found
(x2)

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm

FailedMount

MountVolume.SetUp failed for volume "kube-api-access-l4frg" : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-2lrpd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-196e7e-699dcb9b76-5cnvb

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-196e7e-7785d9947-rfhnn

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc

SuccessfulCreate

Created pod: e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p

openshift-ingress

replicaset-controller

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc

SuccessfulCreate

Created pod: e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x

openshift-ingress

deployment-controller

e2e-worker-w4-196e7e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-196e7e-openshift-default-5b9d665fbc from 2 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-196e7e-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p

Started

Container started

openshift-ingress

multus

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-xhw5l

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vnr4x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-vkvtv

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6kjfp

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-6kjfp

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w2-29e7e8-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-29e7e8-openshift-default-5d7f77986f from 2 to 1

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulDelete

Deleted pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-6l6xg

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target
(x10)

models-as-a-service

kubelet

payload-processing-external-model-5f69dbdb4d-vvrwp

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-ee7ade4978e6" not found
(x33)

models-as-a-service

kubelet

payload-processing-external-model-68df898747-shn7h

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-68df898747-shn7h

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-5f69dbdb4d-vvrwp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dab533-99d78c7cf-mpgds

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dab533-7cb99f96-4hfhh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-a-dab533-openshift-default-6668bd9897-hdl26

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3d082d-7cb999b99b-pknll

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-3d082d-openshift-default-5d8b85969-stxsd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3d082d-746b8ff675-49c2b

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-29e7e8-784c8fdf5-c6q7x

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-29e7e8-db9db6774-nltfn

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

e2e-worker-w2-29e7e8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-29e7e8-openshift-default-5d7f77986f from 1 to 3

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulCreate

Created pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

openshift-ingress

multus

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

AddedInterface

Add eth0 [10.134.0.120/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-29e7e8-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f

SuccessfulCreate

Created pod: e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

openshift-ingress

multus

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

ProbeError

Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

Unhealthy

Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-79cm8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-z8pz7

Killing

Stopping container istio-proxy