Time Namespace Component RelatedObject Reason Message

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-q7hnw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-q7hnw to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

kube-auth-proxy-d555fc4c8-6r5vq

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-d555fc4c8-6r5vq to ip-10-0-136-190.us-west-2.compute.internal

ai-tenants

e2e-extmod-a-80b404

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-80b404: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-3353a6

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-3353a6: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-ab61a8

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-ab61a8: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-d501fd

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-d501fd: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-2ba6b8

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-2ba6b8: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w1-a18c43

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w1-a18c43: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-1e4f1e

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-1e4f1e: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-6c403f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-6c403f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-ab53c1

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-ab53c1: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-7q8zn

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-7q8zn to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-m5ppb

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-m5ppb to ip-10-0-136-190.us-west-2.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-9x7xf

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-9x7xf to ip-10-0-136-190.us-west-2.compute.internal

opendatahub

maas-controller-6b86f7d796-mnw99

Scheduled

Successfully assigned opendatahub/maas-controller-6b86f7d796-mnw99 to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

maas-controller-74b9444797-jtnrf

Scheduled

Successfully assigned opendatahub/maas-controller-74b9444797-jtnrf to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

model-serving-api-75bd89fdfc-phrf7

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-phrf7 to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

odh-model-controller-769875db56-h9lpt

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-h9lpt to ip-10-0-131-226.us-west-2.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-7r89w

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-7r89w to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

test-curl-7507-40f94d

Scheduled

Successfully assigned opendatahub/test-curl-7507-40f94d to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-z99gz

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-z99gz to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

test-curl-7507-ced468

Scheduled

Successfully assigned opendatahub/test-curl-7507-ced468 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-4l68q

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-4l68q to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m to ip-10-0-136-190.us-west-2.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zbm8q to ip-10-0-136-190.us-west-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-8bf8997fb-4crf8 to ip-10-0-131-226.us-west-2.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-6w9wt

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-6w9wt to ip-10-0-131-226.us-west-2.compute.internal

opendatahub

ai-gateway-controller-f6fb8bb7-gkb7n

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-f6fb8bb7-gkb7n to ip-10-0-130-155.us-west-2.compute.internal

opendatahub

ai-gateway-controller-5d8dddb676-5gvws

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-5gvws to ip-10-0-130-155.us-west-2.compute.internal

ai-tenants

e2e-extmod-a-31626a

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-31626a: MaasTenantConfig is not available to restore revocation operands

models-as-a-service

payload-processing-external-model-fc445457d-9qzxp

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-fc445457d-9qzxp to ip-10-0-131-226.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-b6fbcb49-hvml2

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-b6fbcb49-hvml2 to ip-10-0-136-190.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-6945776df8-p4k5q

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6945776df8-p4k5q to ip-10-0-131-226.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-66ccc7f677-72xcm

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-66ccc7f677-72xcm to ip-10-0-131-226.us-west-2.compute.internal

models-as-a-service

payload-processing-external-model-54f9d586c7-9k9pj

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-54f9d586c7-9k9pj to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-hs66c

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-hs66c to ip-10-0-136-190.us-west-2.compute.internal

kuadrant-system

authorino-56c6c9ddd5-5hrdw

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-5hrdw to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4 to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4 to ip-10-0-131-226.us-west-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-8bf8997fb-jckrs to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv to ip-10-0-131-226.us-west-2.compute.internal

kuadrant-system

authorino-57f7f8659f-r9rw9

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-r9rw9 to ip-10-0-130-155.us-west-2.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-z6pgx

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-z6pgx to ip-10-0-136-190.us-west-2.compute.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-lwlzf

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lwlzf to ip-10-0-136-190.us-west-2.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-8bf8997fb-smgkh to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

kube-auth-proxy-d555fc4c8-4mcz9

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-d555fc4c8-4mcz9 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6 to ip-10-0-131-226.us-west-2.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt to ip-10-0-131-226.us-west-2.compute.internal

kuadrant-system

authorino-5bb7848768-52mgq

Scheduled

Successfully assigned kuadrant-system/authorino-5bb7848768-52mgq to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8 to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2 to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78 to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5 to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp to ip-10-0-136-190.us-west-2.compute.internal

kuadrant-system

authorino-6f6944bcdd-d7jz2

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-d7jz2 to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57 to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9 to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-processing-7bb9489676-bd857

Scheduled

Successfully assigned openshift-ingress/payload-processing-7bb9489676-bd857 to ip-10-0-131-226.us-west-2.compute.internal

kuadrant-system

authorino-785f46d47c-6xj7p

Scheduled

Successfully assigned kuadrant-system/authorino-785f46d47c-6xj7p to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-gdn98

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-gdn98 to ip-10-0-131-226.us-west-2.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-snzd9

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-snzd9 to ip-10-0-131-226.us-west-2.compute.internal

ai-tenants

e2e-extmod-a-493b10

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-493b10: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5 to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-wpdjn

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-wpdjn to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-xdcsw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-xdcsw to ip-10-0-131-226.us-west-2.compute.internal

kuadrant-system

authorino-7975ddd498-zgbbp

Scheduled

Successfully assigned kuadrant-system/authorino-7975ddd498-zgbbp to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

router-default-6b8f6d865c-c8tlq

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-6b8f6d865c-c8tlq

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-6b8f6d865c-c8tlq

Scheduled

Successfully assigned openshift-ingress/router-default-6b8f6d865c-c8tlq to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-c8n4k

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-c8n4k to ip-10-0-136-190.us-west-2.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-fswlr

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-fswlr to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-fm8nm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-fm8nm to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-24gct

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-24gct to ip-10-0-136-190.us-west-2.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-ckdtv

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-ckdtv to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-785jm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-785jm to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-kzkx2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-kzkx2 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-jrkdc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-jrkdc to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-7r6mg

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-7r6mg to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-8qbmc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-8qbmc to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-9rmjg

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-9rmjg to ip-10-0-130-155.us-west-2.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-fd4b99879-b8qff

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fd4b99879-b8qff to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-79f98cb86f2b78h

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-79f98cb86f2b78h to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q to ip-10-0-131-226.us-west-2.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-rtvdm

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-rtvdm to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp to ip-10-0-131-226.us-west-2.compute.internal

openshift-jobset-operator

jobset-controller-manager-589bb644c6-flmh9

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-589bb644c6-flmh9 to ip-10-0-130-155.us-west-2.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-b2r9v

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-b2r9v to ip-10-0-130-155.us-west-2.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc to ip-10-0-130-155.us-west-2.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-jcqq8

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-jcqq8 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb to ip-10-0-130-155.us-west-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-qtzcq

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-qtzcq to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5 to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7 to ip-10-0-131-226.us-west-2.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-vkkgd

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-vkkgd to ip-10-0-131-226.us-west-2.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-ll6br

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-ll6br to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2 to ip-10-0-136-190.us-west-2.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-bld85

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-bld85 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2 to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4 to ip-10-0-131-226.us-west-2.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-fcdkx

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-fcdkx to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng to ip-10-0-130-155.us-west-2.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-bkv7h

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-bkv7h to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4 to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b to ip-10-0-136-190.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2 to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w to ip-10-0-130-155.us-west-2.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-f9kjr

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9kjr to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd to ip-10-0-130-155.us-west-2.compute.internal

openshift-ingress

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz to ip-10-0-131-226.us-west-2.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-6b8f6d865c from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-6b8f6d865c

SuccessfulCreate

Created pod: router-default-6b8f6d865c-c8tlq
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x6)

openshift-ingress

kubelet

router-default-6b8f6d865c-c8tlq

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x6)

openshift-ingress

kubelet

router-default-6b8f6d865c-c8tlq

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

multus

router-default-6b8f6d865c-c8tlq

AddedInterface

Add eth0 [10.132.0.9/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-6b8f6d865c-c8tlq

Created

Container created

openshift-ingress

kubelet

router-default-6b8f6d865c-c8tlq

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

router-default-6b8f6d865c-c8tlq

Started

Container started

openshift-ingress

kubelet

router-default-6b8f6d865c-c8tlq

ProbeError

Startup probe error: HTTP probe failed with statuscode: 500 body: [-]backend-proxy-http failed: reason withheld [+]has-synced ok [+]process-running ok healthz check failed

openshift-ingress

kubelet

router-default-6b8f6d865c-c8tlq

Unhealthy

Startup probe failed: HTTP probe failed with statuscode: 500

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-hs66c

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-hs66c

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-hs66c

AddedInterface

Add eth0 [10.132.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-hs66c

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-hs66c

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-hs66c

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.468s (4.468s including waiting). Image size: 185476477 bytes.
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-hs66c

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-hs66c

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-fd4b99879 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-b8qff

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-666dbc6559

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-666dbc6559 from 0 to 1

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-b8qff

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 14.9s (14.9s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w

Started

Container started

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 15.917s (15.917s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-666dbc6559-sdd6w

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-bld85

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-bld85

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked
(x4)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-bld85

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found
(x9)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-7r89w

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-7r89w

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-bld85

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-bld85

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-bld85

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 5.415s (5.415s including waiting). Image size: 224874686 bytes.

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-7r89w

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-bld85_ddab3182-4244-4cc3-9c9c-1b9a1f3270f8 became leader

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-b2r9v

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-b2r9v

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-b2r9v

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-6w9wt

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-6w9wt

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-6w9wt

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install
(x12)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-6w9wt

Created

Container created

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-6w9wt

Started

Container started

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-7r89w

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 6.149s (6.149s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-7r89w

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-7r89w

Created

Container created

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-6w9wt

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 1.591s (1.591s including waiting). Image size: 309650901 bytes.

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-fswlr

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-b2r9v

Created

Container created

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-8bf8997fb from 0 to 3

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-b2r9v

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 5.473s (5.473s including waiting). Image size: 177967631 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-b2r9v

Started

Container started

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

multus

authorino-operator-7b7bb4bc77-fswlr

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fswlr

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-8bf8997fb

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-8bf8997fb-jckrs

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-8bf8997fb-smgkh

AddedInterface

Add eth0 [10.132.0.26/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-8bf8997fb-jckrs

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-8bf8997fb

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-8bf8997fb-4crf8

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-8bf8997fb

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-8bf8997fb-smgkh

kuadrant-system

limitador-operator-controller-manager-5778bb4449-b2r9v_b5618fca-c4b1-48a6-a203-08deb4339e54

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-b2r9v_b5618fca-c4b1-48a6-a203-08deb4339e54 became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-8bf8997fb-4crf8

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fswlr

Started

Container started

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fswlr

Created

Container created

kuadrant-system

authorino-operator-7b7bb4bc77-fswlr_9c2aa5f5-8c0b-4b47-823f-46abe02d19df

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-fswlr_9c2aa5f5-8c0b-4b47-823f-46abe02d19df became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-fswlr

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 2.806s (2.806s including waiting). Image size: 167738465 bytes.

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-ll6br

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-ll6br

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-ll6br

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.215s (4.215s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.044s (4.044s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 5.003s (5.003s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 860ms (860ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-smgkh

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-ll6br

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-ll6br

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 3.373s (3.373s including waiting). Image size: 190804149 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 945ms (945ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-ll6br

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-jckrs

Started

Container started

kuadrant-system

dns-operator-controller-manager-8557cbdb84-ll6br_c7934540-71dc-4c6e-bef3-ec5d6d1f0a0c

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-ll6br_c7934540-71dc-4c6e-bef3-ec5d6d1f0a0c became leader

opendatahub-operator-system

opendatahub-operator-controller-manager-8bf8997fb-smgkh_e4c66dd4-8294-4aaf-996f-76c84d5095bc

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-8bf8997fb-smgkh_e4c66dd4-8294-4aaf-996f-76c84d5095bc became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 846ms (846ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Created

Container created

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-8bf8997fb-4crf8

Started

Container started

cert-manager

multus

cert-manager-597b95cb84-7q8zn

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-7q8zn

Created

Container created

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-qtzcq

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-7q8zn

cert-manager

kubelet

cert-manager-597b95cb84-7q8zn

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

cert-manager

kubelet

cert-manager-597b95cb84-7q8zn

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-qtzcq

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-qtzcq

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-qtzcq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 6.437s (6.437s including waiting). Image size: 424244516 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-qtzcq

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-qtzcq

Created

Container created

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-qtzcq_7b2088c0-6478-4fd7-b687-9b9abb753bda

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-qtzcq_7b2088c0-6478-4fd7-b687-9b9abb753bda became leader

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-bkv7h

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-bkv7h

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-bkv7h

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-f9kjr

Started

Container started

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-f9kjr

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-f9kjr

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-f9kjr

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-f9kjr

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-bkv7h

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 7.775s (7.775s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-bkv7h

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-bkv7h

Started

Container started

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-d555fc4c8 from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-d555fc4c8

SuccessfulCreate

Created pod: kube-auth-proxy-d555fc4c8-4mcz9

openshift-ingress

replicaset-controller

kube-auth-proxy-d555fc4c8

SuccessfulCreate

Created pod: kube-auth-proxy-d555fc4c8-6r5vq

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-79f98cb86f2b78h

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-79f98cb86f from 0 to 1

openshift-ingress

multus

kube-auth-proxy-d555fc4c8-6r5vq

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-79f98cb86f

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-79f98cb86f2b78h

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-6r5vq

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-d555fc4c8-4mcz9

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-4mcz9

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-79f98cb86f2b78h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-jcqq8

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-79f98cb86f2b78h

Created

Container created

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-79f98cb86f2b78h

Started

Container started

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-qtzcq

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jcqq8

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

multus

limitador-limitador-74dfcfb894-jcqq8

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-qtzcq

Killing

Stopping container manager

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-6r5vq

Started

Container started

kuadrant-system

multus

authorino-57f7f8659f-r9rw9

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-6r5vq

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-6r5vq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.43s (3.43s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-4mcz9

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-4mcz9

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-d555fc4c8-4mcz9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.149s (3.149s including waiting). Image size: 188904677 bytes.

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-r9rw9

kuadrant-system

kubelet

authorino-57f7f8659f-r9rw9

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-cqf62"

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-f9kr2"

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jcqq8

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 3.701s (3.701s including waiting). Image size: 133379307 bytes.

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jcqq8

Started

Container started

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jcqq8

Created

Container created

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-589bb644c6

SuccessfulCreate

Created pod: jobset-controller-manager-589bb644c6-flmh9

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

kuadrant-system

kubelet

authorino-57f7f8659f-r9rw9

Started

Container started

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-589bb644c6 from 0 to 1

kuadrant-system

kubelet

authorino-57f7f8659f-r9rw9

Created

Container created

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

kuadrant-system

kubelet

authorino-57f7f8659f-r9rw9

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 4.259s (4.259s including waiting). Image size: 204096810 bytes.

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

openshift-jobset-operator

kubelet

jobset-controller-manager-589bb644c6-flmh9

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

openshift-jobset-operator

multus

jobset-controller-manager-589bb644c6-flmh9

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-9x7xf

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

openshift-jobset-operator

kubelet

jobset-controller-manager-589bb644c6-flmh9

Started

Container started

openshift-jobset-operator

kubelet

jobset-controller-manager-589bb644c6-flmh9

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-589bb644c6-flmh9

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 4.565s (4.565s including waiting). Image size: 182184678 bytes.

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-9x7xf

AddedInterface

Add eth0 [10.132.0.28/23] from ovn-kubernetes

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-9x7xf

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

openshift-jobset-operator

jobset-controller-manager-589bb644c6-flmh9_d4566617-dbae-4630-bde3-fb6adef49bd9

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-589bb644c6-flmh9_d4566617-dbae-4630-bde3-fb6adef49bd9 became leader

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-9x7xf

Started

Container started

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-9x7xf

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.143s (2.143s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-9x7xf

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-f9kjr_e5021521-faad-488e-9232-99a1ff252de5

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-f9kjr_e5021521-faad-488e-9232-99a1ff252de5 became leader

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-ckdtv

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-5hrdw

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-ckdtv

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-ckdtv

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-5hrdw

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-56c6c9ddd5-5hrdw

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

kuadrant-system

multus

limitador-limitador-7c9b87d574-ckdtv

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-ckdtv

Pulled

Container image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

authorino-56c6c9ddd5-5hrdw

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.537s (3.537s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-5hrdw

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

authorino-56c6c9ddd5-5hrdw

Created

Container created

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

authorino-57f7f8659f-r9rw9

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-r9rw9

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kserve-module-controller-manager-7566cf5fc8-9x7xf_32efdac0-7642-466f-ad1d-d90440297ab4

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-9x7xf_32efdac0-7642-466f-ad1d-d90440297ab4 became leader

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-jcqq8

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jcqq8

Killing

Stopping container limitador

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-lwlzf

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-lwlzf

Started

Container started

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-lwlzf

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-lwlzf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-lwlzf

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-lwlzf

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-lwlzf

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-hs66c

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-hs66c

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-z99gz

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-m5ppb

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-phrf7

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-h9lpt

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

kubelet

kserve-controller-manager-776cc66d47-z99gz

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

multus

kserve-controller-manager-776cc66d47-z99gz

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-m5ppb

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

opendatahub

multus

model-serving-api-75bd89fdfc-phrf7

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-phrf7

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

kubelet

odh-model-controller-769875db56-h9lpt

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

multus

odh-model-controller-769875db56-h9lpt

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-m5ppb

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

replicaset-controller

maas-controller-6b86f7d796

SuccessfulCreate

Created pod: maas-controller-6b86f7d796-mnw99

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-6b86f7d796 from 0 to 1

opendatahub

kubelet

maas-controller-74b9444797-jtnrf

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:2c21ffc3b7b4d3af0789e6704042292256f5a3937596a0076af8fb3689d1f1a9"

opendatahub

kubelet

maas-controller-6b86f7d796-mnw99

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:2c21ffc3b7b4d3af0789e6704042292256f5a3937596a0076af8fb3689d1f1a9"

opendatahub

kubelet

model-serving-api-75bd89fdfc-phrf7

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-phrf7

Created

Container created

opendatahub

multus

maas-controller-6b86f7d796-mnw99

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

opendatahub

replicaset-controller

maas-controller-74b9444797

SuccessfulCreate

Created pod: maas-controller-74b9444797-jtnrf

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-m5ppb

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-phrf7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 3.13s (3.13s including waiting). Image size: 184925761 bytes.

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-74b9444797 from 0 to 1

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-m5ppb

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-m5ppb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.908s (2.908s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-z99gz

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.735s (3.735s including waiting). Image size: 247676581 bytes.

opendatahub

multus

maas-controller-74b9444797-jtnrf

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

opendatahub

kubelet

kserve-controller-manager-776cc66d47-z99gz

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-z99gz

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-h9lpt

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 4.5s (4.5s including waiting). Image size: 236755567 bytes.

opendatahub

llmisvc-controller-manager-869d5dcc5d-m5ppb_3cd3f3df-d6c5-4dce-9616-ac503a0ea8ab

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-m5ppb_3cd3f3df-d6c5-4dce-9616-ac503a0ea8ab became leader

opendatahub

kserve-controller-manager-776cc66d47-z99gz_542bc691-8a64-4dc0-978e-6d7205b8b710

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-z99gz_542bc691-8a64-4dc0-978e-6d7205b8b710 became leader

opendatahub

kubelet

odh-model-controller-769875db56-h9lpt

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-h9lpt

Created

Container created

opendatahub

odh-model-controller-769875db56-h9lpt_064035c6-9199-4062-9bcc-26d47c752582

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-h9lpt_064035c6-9199-4062-9bcc-26d47c752582 became leader

opendatahub

kubelet

maas-controller-74b9444797-jtnrf

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:2c21ffc3b7b4d3af0789e6704042292256f5a3937596a0076af8fb3689d1f1a9" in 5.941s (5.941s including waiting). Image size: 200103441 bytes.

opendatahub

kubelet

maas-controller-74b9444797-jtnrf

Created

Container created

opendatahub

kubelet

maas-controller-74b9444797-jtnrf

Started

Container started

opendatahub

kubelet

maas-controller-6b86f7d796-mnw99

Started

Container started

opendatahub

kubelet

maas-controller-6b86f7d796-mnw99

Created

Container created

opendatahub

kubelet

maas-controller-6b86f7d796-mnw99

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:2c21ffc3b7b4d3af0789e6704042292256f5a3937596a0076af8fb3689d1f1a9" in 6.489s (6.489s including waiting). Image size: 200103441 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

maas-controller-6b86f7d796-mnw99_acbe630c-dc04-4580-9c60-2a937636bced

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-6b86f7d796-mnw99_acbe630c-dc04-4580-9c60-2a937636bced became leader

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

FailedMount

MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

FailedMount

MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-6b86f7d796-mnw99

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-6b86f7d796

SuccessfulDelete

Deleted pod: maas-controller-6b86f7d796-mnw99

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-6b86f7d796 from 1 to 0
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 18.287s (18.287s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-zbm8q

Started

Container started

opendatahub

maas-controller-74b9444797-jtnrf_a47ca1e4-1913-4a2e-a73d-87045ec917e2

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-74b9444797-jtnrf_a47ca1e4-1913-4a2e-a73d-87045ec917e2 became leader

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

replicaset-controller

authorino-5bb7848768

SuccessfulCreate

Created pod: authorino-5bb7848768-52mgq

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-5bb7848768 from 0 to 1

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-d7jz2

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-d7jz2

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-7975ddd498 from 0 to 1

kuadrant-system

replicaset-controller

authorino-5bb7848768

SuccessfulDelete

Deleted pod: authorino-5bb7848768-52mgq

kuadrant-system

multus

authorino-5bb7848768-52mgq

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-5bb7848768-52mgq

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-5bb7848768 from 1 to 0

kuadrant-system

replicaset-controller

authorino-7975ddd498

SuccessfulCreate

Created pod: authorino-7975ddd498-zgbbp

kuadrant-system

kubelet

authorino-6f6944bcdd-d7jz2

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-6f6944bcdd-d7jz2

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-7975ddd498-zgbbp

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

Created

Container created
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

multus

authorino-7975ddd498-zgbbp

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-5bb7848768-52mgq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.025s (3.025s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-5bb7848768-52mgq

Created

Container created

kuadrant-system

kubelet

authorino-5bb7848768-52mgq

Started

Container started

kuadrant-system

kubelet

authorino-7975ddd498-zgbbp

Started

Container started

kuadrant-system

kubelet

authorino-7975ddd498-zgbbp

Created

Container created

kuadrant-system

kubelet

authorino-7975ddd498-zgbbp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.662s (2.662s including waiting). Image size: 204096810 bytes.

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-5hrdw

kuadrant-system

kubelet

authorino-5bb7848768-52mgq

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-56c6c9ddd5-5hrdw

Killing

Stopping container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

ProbeError

Startup probe error: Get "http://10.133.0.24:15021/healthz/ready": dial tcp 10.133.0.24:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

Unhealthy

Startup probe failed: Get "http://10.133.0.24:15021/healthz/ready": dial tcp 10.133.0.24:15021: connect: connection refused

kuadrant-system

kubelet

authorino-6f6944bcdd-d7jz2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 7.512s (7.512s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6f6944bcdd-d7jz2

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-d7jz2

Started

Container started

kuadrant-system

kubelet

authorino-6f6944bcdd-d7jz2

Killing

Stopping container authorino
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-z6pgx

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

multus

llmisvc-controller-manager-6466c554d8-z6pgx

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-z6pgx

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-z6pgx

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-z6pgx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 795ms (795ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-z6pgx

Created

Container created

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-m5ppb

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-m5ppb

Killing

Stopping container manager

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

multus

ai-gateway-controller-5d8dddb676-5gvws

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-5gvws

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-5gvws

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-5gvws

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-5gvws

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 3.329s (3.329s including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-5gvws

Created

Container created

opendatahub

ai-gateway-controller-5d8dddb676-5gvws_584b4441-66bc-42b5-a059-c41637bd791c

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-5gvws_584b4441-66bc-42b5-a059-c41637bd791c became leader

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-4l68q
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

SuccessfulCreate

Created pod: payload-processing-7bb9489676-bd857
(x7)

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

FailedCreate

Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-7bb9489676 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

multus

payload-pre-processing-7b98bb678c-4l68q

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-7bb9489676-bd857

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-7bb9489676-bd857

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-4l68q

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-7bb9489676-bd857

Started

Container started

openshift-ingress

kubelet

payload-processing-7bb9489676-bd857

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.539s (4.539s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-4l68q

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 4.75s (4.75s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-4l68q

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-4l68q

Started

Container started

openshift-ingress

kubelet

payload-processing-7bb9489676-bd857

Created

Container created

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-f6fb8bb7 from 0 to 1

opendatahub

kubelet

ai-gateway-controller-f6fb8bb7-gkb7n

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

multus

ai-gateway-controller-f6fb8bb7-gkb7n

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

opendatahub

replicaset-controller

ai-gateway-controller-f6fb8bb7

SuccessfulCreate

Created pod: ai-gateway-controller-f6fb8bb7-gkb7n

opendatahub

llmisvc-controller-manager-6466c554d8-z6pgx_7f58b4ee-45e4-4655-9ade-91a1b5d5edad

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-z6pgx_7f58b4ee-45e4-4655-9ade-91a1b5d5edad became leader

opendatahub

kubelet

ai-gateway-controller-f6fb8bb7-gkb7n

Started

Container started

opendatahub

kubelet

ai-gateway-controller-f6fb8bb7-gkb7n

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 925ms (925ms including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-f6fb8bb7-gkb7n

Created

Container created

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-5gvws

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-5gvws

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.288s (2.288s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-9fhrc

Created

Container created

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-vkkgd

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-vkkgd

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vkkgd

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

opendatahub

ai-gateway-controller-f6fb8bb7-gkb7n_25016836-9d9c-45a6-8751-f87924ee995d

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-f6fb8bb7-gkb7n_25016836-9d9c-45a6-8751-f87924ee995d became leader

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vkkgd

Started

Container started

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vkkgd

Created

Container created

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-vkkgd

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.533s (1.533s including waiting). Image size: 98346788 bytes.

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-fcdkx

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-fcdkx

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-fcdkx

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 573ms (573ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-fcdkx

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-fcdkx

Started

Container started

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-fcdkx

Created

Container created

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-rtvdm

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-rtvdm

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-rtvdm

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-rtvdm

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 553ms (553ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-rtvdm

Created

Container created

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-rtvdm

Started

Container started

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 643ms (643ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pbxm9

Started

Container started

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-snzd9

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

kubelet

echo-simulator-kserve-64b6f8f96c-snzd9

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

multus

echo-simulator-kserve-64b6f8f96c-snzd9

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd

llm

kubelet

echo-simulator-kserve-64b6f8f96c-snzd9

Started

Container started

llm

kubelet

echo-simulator-kserve-64b6f8f96c-snzd9

Created

Container created

llm

kubelet

echo-simulator-kserve-64b6f8f96c-snzd9

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.65s (2.65s including waiting). Image size: 78208878 bytes.

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd

Started

Container started

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-wqtpd

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 574ms (574ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-fd4b99879 from 1 to 3

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-24gct

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-kzkx2

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-24gct

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-kzkx2

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 581ms (581ms including waiting). Image size: 98346788 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-24gct

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-kzkx2

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-kzkx2

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.341s (4.341s including waiting). Image size: 183555333 bytes.

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-kzkx2

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-24gct

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-kzkx2

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-24gct

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-24gct

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.296s (4.296s including waiting). Image size: 183555333 bytes.

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-24gct

ProbeError

Startup probe error: Get "http://10.132.0.35:15021/healthz/ready": dial tcp 10.132.0.35:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-24gct

Unhealthy

Startup probe failed: Get "http://10.132.0.35:15021/healthz/ready": dial tcp 10.132.0.35:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-kzkx2

ProbeError

Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-kzkx2

Unhealthy

Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-9rmjg

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-fd4b99879 from 3 to 6

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-jrkdc

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-7r6mg

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-7r6mg

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-jrkdc

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-9rmjg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-9rmjg

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-7r6mg

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-7r6mg

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-7r6mg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-jrkdc

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-9rmjg

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-9rmjg

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-jrkdc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-jrkdc

Created

Container created

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-jrkdc

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-jrkdc

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-9rmjg

ProbeError

Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-7r6mg

Unhealthy

Startup probe failed: Get "http://10.132.0.36:15021/healthz/ready": dial tcp 10.132.0.36:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-7r6mg

ProbeError

Startup probe error: Get "http://10.132.0.36:15021/healthz/ready": dial tcp 10.132.0.36:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-9rmjg

Unhealthy

Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

kuadrant-system

replicaset-controller

authorino-785f46d47c

SuccessfulCreate

Created pod: authorino-785f46d47c-6xj7p

kuadrant-system

kubelet

authorino-785f46d47c-6xj7p

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-785f46d47c-6xj7p

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-785f46d47c-6xj7p

Started

Container started
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-7975ddd498 from 1 to 0

kuadrant-system

kubelet

authorino-7975ddd498-zgbbp

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-785f46d47c-6xj7p

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.647s (1.647s including waiting). Image size: 204096810 bytes.

kuadrant-system

replicaset-controller

authorino-7975ddd498

SuccessfulDelete

Deleted pod: authorino-7975ddd498-zgbbp

kuadrant-system

kubelet

authorino-785f46d47c-6xj7p

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5

SuccessfulCreate

Created pod: e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-1e4f1e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-1e4f1e

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-6c403f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-6c403f

openshift-ingress

deployment-controller

e2e-worker-w2-1e4f1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-ab53c1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-ab53c1

openshift-ingress

deployment-controller

e2e-worker-w3-6c403f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-6c403f-openshift-default-78c4b9f87 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w3-6c403f-openshift-default-78c4b9f87

SuccessfulCreate

Created pod: e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

openshift-ingress

multus

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-2ba6b8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-2ba6b8

openshift-ingress

replicaset-controller

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c

SuccessfulCreate

Created pod: e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

openshift-ingress

deployment-controller

e2e-worker-w0-2ba6b8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c from 0 to 1

openshift-ingress

multus

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w1-a18c43

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-a18c43

openshift-ingress

deployment-controller

e2e-worker-w1-a18c43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-a18c43-openshift-default-797b4855dc from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-ab53c1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-ab53c1: serviceaccount "payload-processing-e2e-worker-w4-ab53c1" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-ab53c1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-ab53c1: serviceaccount "payload-processing-e2e-worker-w4-ab53c1" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-6c403f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-6c403f-f697477cb from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-6c403f: serviceaccount "payload-processing-e2e-worker-w3-6c403f" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-6c403f-f697477cb

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-6c403f-f697477cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-6c403f: serviceaccount "payload-processing-e2e-worker-w3-6c403f" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-6c403f-f697477cb

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-6c403f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-6c403f-66f8855447 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-1e4f1e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-1e4f1e-57b677d95d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-1e4f1e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-1e4f1e: serviceaccount "payload-processing-e2e-worker-w2-1e4f1e" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-1e4f1e: serviceaccount "payload-processing-e2e-worker-w2-1e4f1e" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp

openshift-ingress

multus

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp

Started

Container started

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2ba6b8: serviceaccount "payload-processing-e2e-worker-w0-2ba6b8" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-2ba6b8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-2ba6b8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-2ba6b8: serviceaccount "payload-processing-e2e-worker-w0-2ba6b8" not found

openshift-ingress

multus

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.924s (2.924s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.125s (3.125s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w1-a18c43

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-a18c43: serviceaccount "payload-processing-e2e-worker-w1-a18c43" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w1-a18c43

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w1-a18c43-64f77d44d from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-a18c43-64f77d44d

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w1-a18c43-64f77d44d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-a18c43: serviceaccount "payload-processing-e2e-worker-w1-a18c43" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-a18c43-64f77d44d

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

Unhealthy

Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

ProbeError

Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-6c403f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-6c403f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

Unhealthy

Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

ProbeError

Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

Unhealthy

Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

ProbeError

Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-6c403f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-6c403f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

Created

Container created
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2ba6b8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2ba6b8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

ProbeError

Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

Unhealthy

Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2ba6b8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2ba6b8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-worker-w2-1e4f1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5 from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5

SuccessfulCreate

Created pod: e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

Started

Container started

openshift-ingress

multus

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w1-a18c43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-a18c43-openshift-default-797b4855dc from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

Unhealthy

Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

ProbeError

Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused body:

openshift-ingress

multus

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c

SuccessfulCreate

Created pod: e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

ProbeError

Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

ProbeError

Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

Unhealthy

Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused

opendatahub

multus

test-curl-7507-ced468

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7507-ced468

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-24gct

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-b8qff

Killing

Stopping container istio-proxy

opendatahub

kubelet

test-curl-7507-ced468

Started

Container started

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-fd4b99879-b8qff

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-fd4b99879-24gct

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-fd4b99879 from 6 to 4

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

opendatahub

kubelet

test-curl-7507-ced468

Created

Container created

opendatahub

kubelet

test-curl-7507-ced468

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.441s (2.441s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7507-40f94d

Started

Container started

opendatahub

kubelet

test-curl-7507-ced468

Killing

Stopping container test-curl-7507-ced468

opendatahub

multus

test-curl-7507-40f94d

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7507-40f94d

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7507-40f94d

Created

Container created

opendatahub

kubelet

test-curl-7507-40f94d

Killing

Stopping container test-curl-7507-40f94d

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-493b10

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-493b10

openshift-ingress

deployment-controller

e2e-extmod-a-493b10-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-493b10-openshift-default-c47476b77 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-493b10-openshift-default-c47476b77

SuccessfulCreate

Created pod: e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

openshift-ingress

multus

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w2-1e4f1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5 from 2 to 3

openshift-ingress

replicaset-controller

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5

SuccessfulCreate

Created pod: e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

ProbeError

Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

Unhealthy

Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-493b10

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-493b10

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-493b10-54d94b558 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-493b10-54d94b558

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-493b10-54d94b558-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-493b10: serviceaccount "payload-processing-e2e-extmod-a-493b10" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-493b10: serviceaccount "payload-processing-e2e-extmod-a-493b10" not found

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-493b10-54d94b558

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-493b10-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-493b10-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-extmod-a-31626a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-31626a-openshift-default-78dc97674f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-31626a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-31626a

openshift-ingress

replicaset-controller

e2e-extmod-a-31626a-openshift-default-78dc97674f

SuccessfulCreate

Created pod: e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

openshift-ingress

multus

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-31626a: serviceaccount "payload-processing-e2e-extmod-a-31626a" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-31626a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-31626a-595f9f84db from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-31626a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-31626a-55f9577c87 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-31626a-595f9f84db

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-31626a-595f9f84db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-31626a: serviceaccount "payload-processing-e2e-extmod-a-31626a" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-31626a-595f9f84db

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w1-a18c43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-a18c43-openshift-default-797b4855dc from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

Started

Container started

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

Unhealthy

Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

ProbeError

Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": read tcp 10.133.0.2:42464->10.133.0.67:15021: read: connection reset by peer body:

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

Unhealthy

Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": read tcp 10.133.0.2:42464->10.133.0.67:15021: read: connection reset by peer
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

ProbeError

Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-fd4b99879 from 4 to 3

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-fd4b99879-jrkdc

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-jrkdc

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

ProbeError

Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

Unhealthy

Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-extmod-a-80b404-openshift-default-78c74d5ccc

SuccessfulCreate

Created pod: e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-80b404

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-80b404

openshift-ingress

deployment-controller

e2e-extmod-a-80b404-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-80b404-openshift-default-78c74d5ccc from 0 to 1

openshift-ingress

multus

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

AddedInterface

Add eth0 [10.133.0.71/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 2 to 3

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-80b404

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-80b404-595754f4c5 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-80b404-67bc9766d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-80b404-67bc9766d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-80b404: serviceaccount "payload-processing-e2e-extmod-a-80b404" not found
(x5)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-80b404: serviceaccount "payload-processing-e2e-extmod-a-80b404" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-80b404

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-80b404-67bc9766d from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-80b404-67bc9766d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-493b10-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-493b10-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5

SuccessfulCreate

Created pod: e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

openshift-ingress

deployment-controller

e2e-worker-w2-1e4f1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5 from 3 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

Started

Container started

openshift-ingress

multus

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

Created

Container created

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-ab61a8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-ab61a8

openshift-ingress

replicaset-controller

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9

SuccessfulCreate

Created pod: e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

openshift-ingress

deployment-controller

e2e-extmod-b-ab61a8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9 from 0 to 1

openshift-ingress

multus

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-kzkx2

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

ProbeError

Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-fd4b99879 from 3 to 2
(x6)

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

Unhealthy

Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-fd4b99879-kzkx2

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-31626a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-31626a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-ab61a8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-ab61a8-859f89f868 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-ab61a8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-ab61a8-859f89f868

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-ab61a8-859f89f868

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-ab61a8-859f89f868-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-ab61a8: serviceaccount "payload-processing-e2e-extmod-b-ab61a8" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-ab61a8: serviceaccount "payload-processing-e2e-extmod-b-ab61a8" not found
(x5)

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

Unhealthy

Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

ProbeError

Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w1-a18c43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-a18c43-openshift-default-797b4855dc from 3 to 5

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

AddedInterface

Add eth0 [10.133.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

Unhealthy

Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

ProbeError

Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

Unhealthy

Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

ProbeError

Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused body:

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-fc445457d from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-fc445457d

SuccessfulCreate

Created pod: payload-processing-external-model-fc445457d-9qzxp
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-fc445457d

FailedCreate

Error creating: pods "payload-processing-external-model-fc445457d-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

multus

payload-processing-external-model-fc445457d-9qzxp

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-fc445457d-9qzxp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-fc445457d-9qzxp

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-fc445457d-9qzxp

Started

Container started

openshift-ingress

deployment-controller

e2e-extmod-b-d501fd-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-d501fd-openshift-default-7946556545 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-d501fd

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-d501fd

openshift-ingress

replicaset-controller

e2e-extmod-b-d501fd-openshift-default-7946556545

SuccessfulCreate

Created pod: e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

openshift-ingress

multus

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-d501fd-8546966659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-d501fd-6dd79696c4

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-d501fd-6dd79696c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-d501fd: serviceaccount "payload-processing-e2e-extmod-b-d501fd" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-d501fd-8546966659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-d501fd-8546966659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-d501fd: serviceaccount "payload-processing-e2e-extmod-b-d501fd" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-d501fd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-d501fd-8546966659 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-d501fd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-d501fd-6dd79696c4 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-d501fd-6dd79696c4

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-66ccc7f677

SuccessfulCreate

Created pod: payload-processing-external-model-66ccc7f677-72xcm

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-66ccc7f677 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-xdcsw

models-as-a-service

kubelet

payload-processing-external-model-66ccc7f677-72xcm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-fd4b99879 from 2 to 4

models-as-a-service

kubelet

payload-processing-external-model-66ccc7f677-72xcm

Created

Container created

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-fm8nm

models-as-a-service

kubelet

payload-processing-external-model-66ccc7f677-72xcm

Started

Container started

models-as-a-service

multus

payload-processing-external-model-66ccc7f677-72xcm

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-fm8nm

AddedInterface

Add eth0 [10.133.0.75/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-xdcsw

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-xdcsw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-xdcsw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-xdcsw

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-fm8nm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-31626a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-fm8nm

Created

Container created
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-31626a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-fm8nm

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 3 to 6

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

Created

Container created
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-xdcsw

Unhealthy

Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-xdcsw

ProbeError

Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

ProbeError

Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

Started

Container started

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Unhealthy

Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

Started

Container started
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-fm8nm

ProbeError

Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-fm8nm

Unhealthy

Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused

models-as-a-service

kubelet

payload-processing-external-model-fc445457d-9qzxp

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-fc445457d from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-fc445457d

SuccessfulDelete

Deleted pod: payload-processing-external-model-fc445457d-9qzxp
(x2)

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

ProbeError

Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

Unhealthy

Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-785jm

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-fd4b99879 from 4 to 7
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

Unhealthy

Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

ProbeError

Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-fd4b99879-c8n4k

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-q7hnw

AddedInterface

Add eth0 [10.133.0.78/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-c8n4k

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-c8n4k

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

Unhealthy

Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

ProbeError

Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-785jm

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-q7hnw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-785jm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-q7hnw

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-c8n4k

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-c8n4k

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-q7hnw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-785jm

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-80b404-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Unhealthy

Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

ProbeError

Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-80b404-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

SuccessfulRescale

New size: 7; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 6 to 7
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-785jm

Unhealthy

Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-785jm

ProbeError

Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-c8n4k

ProbeError

Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-c8n4k

Unhealthy

Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-q7hnw

ProbeError

Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-q7hnw

Unhealthy

Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused

models-as-a-service

multus

payload-processing-external-model-6945776df8-p4k5q

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-6945776df8

SuccessfulCreate

Created pod: payload-processing-external-model-6945776df8-p4k5q

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6945776df8 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-6945776df8-p4k5q

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-6945776df8-p4k5q

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-6945776df8-p4k5q

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

Unhealthy

Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

ProbeError

Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-q7hnw

ProbeError

Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-q7hnw

Unhealthy

Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-c8n4k

Unhealthy

Readiness probe failed: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-c8n4k

ProbeError

Readiness probe error: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-3353a6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-3353a6

openshift-ingress

replicaset-controller

e2e-extmod-b-3353a6-openshift-default-5b5476986f

SuccessfulCreate

Created pod: e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

openshift-ingress

deployment-controller

e2e-extmod-b-3353a6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-3353a6-openshift-default-5b5476986f from 0 to 1

openshift-ingress

multus

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

Killing

Stopping container payload-pre-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-80b404-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-80b404-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-3353a6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-3353a6-56549f6c84 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-3353a6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-3353a6-95945dd67 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-3353a6: serviceaccount "payload-processing-e2e-extmod-b-3353a6" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-3353a6-56549f6c84

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-3353a6-56549f6c84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-3353a6: serviceaccount "payload-processing-e2e-extmod-b-3353a6" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-3353a6-56549f6c84

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

Unhealthy

Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

ProbeError

Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-ab61a8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-ab61a8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target
(x5)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-fd4b99879

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-fd4b99879-8qbmc

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-fd4b99879 from 7 to 10

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-8qbmc

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-wpdjn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-wpdjn

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-wpdjn

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-wpdjn

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-8qbmc

AddedInterface

Add eth0 [10.133.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-8qbmc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-gdn98

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-gdn98

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-gdn98

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-fd4b99879-gdn98

AddedInterface

Add eth0 [10.134.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-8qbmc

Started

Container started
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-a18c43-64f77d44d-s8gjd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-a18c43-c6cc7f675-f88w4

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-gdn98

Unhealthy

Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-wpdjn

Unhealthy

Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-gdn98

ProbeError

Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-wpdjn

ProbeError

Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-8qbmc

ProbeError

Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-8qbmc

Unhealthy

Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-ab61a8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-ab61a8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3353a6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3353a6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d501fd-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d501fd-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

openshift-ingress

deployment-controller

e2e-worker-w1-a18c43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-a18c43-openshift-default-797b4855dc from 5 to 8

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

AddedInterface

Add eth0 [10.134.0.82/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

Created

Container created

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

AddedInterface

Add eth0 [10.133.0.83/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

Unhealthy

Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

ProbeError

Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

ProbeError

Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

Unhealthy

Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

Unhealthy

Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

ProbeError

Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

ProbeError

Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

Unhealthy

Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dcvnj

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-x5dz4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4cvtv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4s2mv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-sxcxh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w1-a18c43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-a18c43-openshift-default-797b4855dc from 8 to 10

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-a18c43-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-bp8n5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6

AddedInterface

Add eth0 [10.134.0.83/23] from ovn-kubernetes
(x3)

openshift-ingress

replicaset-controller

e2e-worker-w1-a18c43-openshift-default-797b4855dc

FailedCreate

Error creating: pods "e2e-worker-w1-a18c43-openshift-default-797b4855dc-" is forbidden: error looking up service account openshift-ingress/e2e-worker-w1-a18c43-openshift-default: serviceaccount "e2e-worker-w1-a18c43-openshift-default" not found

openshift-ingress

multus

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-dhmzp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6

Started

Container started

openshift-ingress

deployment-controller

e2e-worker-w1-a18c43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-a18c43-openshift-default-797b4855dc from 0 to 1

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-4sll6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-qrh57

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w1-a18c43-openshift-default-797b4855dc-5sff8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d501fd-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-d501fd-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c

SuccessfulDelete

Deleted pod: e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2ba6b8-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w0-2ba6b8-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c from 2 to 1

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

Unhealthy

Readiness probe failed: Get "http://10.132.0.51:15021/healthz/ready": read tcp 10.132.0.2:42506->10.132.0.51:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-cdhbm

ProbeError

Readiness probe error: Get "http://10.132.0.51:15021/healthz/ready": read tcp 10.132.0.2:42506->10.132.0.51:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

ProbeError

Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

Unhealthy

Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3353a6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-3353a6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-493b10-6cfd49f99-jqbxx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-493b10-54d94b558-4tkmw

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q

Killing

Stopping container payload-pre-processing
(x9)

models-as-a-service

kubelet

payload-processing-external-model-66ccc7f677-72xcm

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-a369bc8f7d56" not found
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-31626a-595f9f84db-kvfjz

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-31626a-55f9577c87-4dp9q

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

e2e-worker-w2-1e4f1e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5 from 4 to 3

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5

SuccessfulDelete

Deleted pod: e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-t5s7w

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-a-493b10-openshift-default-c47476b77-69ftr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-ab61a8-859f89f868-l7vgt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-ab61a8-6c47c5594c-8cdzp

Killing

Stopping container payload-pre-processing
(x25)

models-as-a-service

kubelet

payload-processing-external-model-6945776df8-p4k5q

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-31626a-openshift-default-78dc97674f-nkq4d

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-d501fd-8546966659-x56fc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-d501fd-6dd79696c4-pmlk4

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-54f9d586c7 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-6945776df8-p4k5q

Killing

Stopping container payload-processing

models-as-a-service

multus

payload-processing-external-model-54f9d586c7-9k9pj

AddedInterface

Add eth0 [10.133.0.87/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-54f9d586c7-9k9pj

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

models-as-a-service

replicaset-controller

payload-processing-external-model-54f9d586c7

SuccessfulCreate

Created pod: payload-processing-external-model-54f9d586c7-9k9pj

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-6945776df8 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-6945776df8

SuccessfulDelete

Deleted pod: payload-processing-external-model-6945776df8-p4k5q

models-as-a-service

kubelet

payload-processing-external-model-54f9d586c7-9k9pj

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-54f9d586c7-9k9pj

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-54f9d586c7-9k9pj

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.396s (3.397s including waiting). Image size: 125353963 bytes.
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-9rmjg

Unhealthy

Readiness probe failed: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-9rmjg

ProbeError

Readiness probe error: Get "http://10.133.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-7r6mg

Unhealthy

Readiness probe failed: Get "http://10.132.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-fd4b99879-7r6mg

ProbeError

Readiness probe error: Get "http://10.132.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

Unhealthy

Readiness probe failed: Get "https://10.133.0.49:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

openshift-ingress

deployment-controller

e2e-worker-w2-1e4f1e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5 from 3 to 2

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5

SuccessfulDelete

Deleted pod: e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-cwwh5

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-66ccc7f677 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-66ccc7f677

SuccessfulDelete

Deleted pod: payload-processing-external-model-66ccc7f677-72xcm

models-as-a-service

kubelet

payload-processing-external-model-66ccc7f677-72xcm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 7 to 5

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulDelete

Deleted pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-dbd2b

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulDelete

Deleted pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-ab61a8-openshift-default-57c59cc9d9-6rxf5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

Unhealthy

Readiness probe failed: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-vgp2m

ProbeError

Readiness probe error: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulDelete

Deleted pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-zlwnf

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 5 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-b-d501fd-openshift-default-7946556545-2kpf9

Killing

Stopping container istio-proxy

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-b6fbcb49 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-b6fbcb49

SuccessfulCreate

Created pod: payload-processing-external-model-b6fbcb49-hvml2

models-as-a-service

multus

payload-processing-external-model-b6fbcb49-hvml2

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-b6fbcb49-hvml2

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-b6fbcb49-hvml2

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-b6fbcb49-hvml2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-2ba6b8-6dcc87dd8-wv76k

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-2ba6b8-6549d78655-lrshw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-6c403f-f697477cb-c2qsz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-ab53c1-85dc94c9c6-vtzqt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

deployment-controller

e2e-worker-w0-2ba6b8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c from 1 to 2
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-2ba6b8-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c

SuccessfulCreate

Created pod: e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes
(x2)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-s752z

Unhealthy

Readiness probe failed: Get "https://10.133.0.49:8000/ready": context deadline exceeded

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x2)

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-ab53c1-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-5t4d2

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767

SuccessfulCreate

Created pod: e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

openshift-ingress

deployment-controller

e2e-worker-w4-ab53c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-ab53c1-openshift-default-5dc69f9767 from 4 to 8

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w0-2ba6b8-openshift-default-dockercfg-n895s); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-2ba6b8-openshift-default-bf9cd474c-bspg4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

AddedInterface

Add eth0 [10.133.0.94/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

Created

Container created

openshift-ingress

multus

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

AddedInterface

Add eth0 [10.134.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-6c403f-66f8855447-khpln

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

ProbeError

Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

ProbeError

Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

Unhealthy

Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-ab53c1-566d558cbc-cxfz7

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

Unhealthy

Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

Unhealthy

Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

ProbeError

Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w3-6c403f-openshift-default-78c4b9f87-j7vlm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

Unhealthy

Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

ProbeError

Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-8jqkf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-tlqnl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-jgbwf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gccd4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-gz5m2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-q5zd2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-fjnng

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-ab53c1-openshift-default-5dc69f9767-n558v

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x25)

models-as-a-service

kubelet

payload-processing-external-model-b6fbcb49-hvml2

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

e2e-worker-w2-1e4f1e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5 from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-1e4f1e-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5

SuccessfulDelete

Deleted pod: e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-295vb

Killing

Stopping container istio-proxy
(x10)

models-as-a-service

kubelet

payload-processing-external-model-54f9d586c7-9k9pj

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-63866b33446b" not found

models-as-a-service

kubelet

payload-processing-external-model-54f9d586c7-9k9pj

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-b6fbcb49-hvml2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-80b404-67bc9766d-txrxm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-80b404-595754f4c5-brsw4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-80b404-openshift-default-78c74d5ccc-8nzfd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-3353a6-56549f6c84-72t78

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-b-3353a6-openshift-default-5b5476986f-bdf5t

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-3353a6-95945dd67-pkq4c

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-1e4f1e-5455747c5f-nsjh2

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-1e4f1e-57b677d95d-xsmnp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w2-1e4f1e-openshift-default-846f5ff6b5-nzzdz

Killing

Stopping container istio-proxy