Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-tn2cx to ip-10-0-141-97.us-west-1.compute.internal

ai-tenants

e2e-explicit-5d585e

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-explicit-5d585e: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-a12c35

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-a12c35: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-c73135

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-c73135: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-d96a66

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-d96a66: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-dee6aa

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-dee6aa: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-098e4d

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-098e4d: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-2fd4c9

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-2fd4c9: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-7a360a

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-7a360a: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-c6e99c

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-c6e99c: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-a-2ea125

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-a-2ea125: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-b-af8051

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-b-af8051: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-b-5451cb

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-b-5451cb: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-25565930

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-25565930: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-a08eb536

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-a08eb536: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-a-918b8d

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-a-918b8d: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-b-ca7a55

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-b-ca7a55: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w0-afc555

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w0-afc555: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w1-897c87

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w1-897c87: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w0-1c4191

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w0-1c4191: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w1-fd2672

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w1-fd2672: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-single-main-42db1c

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-single-main-42db1c: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-7d1ec3

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-7d1ec3: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-3d70ff

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-3d70ff: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-a48099

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-a48099: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-fztvp

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-fztvp to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-5f68b7bd84-5rtgt

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5f68b7bd84-5rtgt to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-5f68b7bd84-nfg9z

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5f68b7bd84-nfg9z to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

Scheduled

Successfully assigned openshift-ingress/e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk to ip-10-0-129-92.us-west-1.compute.internal

ai-tenants

e2e-api-a-faa8ff

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-a-faa8ff: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-cainjector-5c57bbd4fc-kc9p7

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-kc9p7 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-2jwpv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-2jwpv to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

Scheduled

Successfully assigned openshift-ingress/e2e-ait-3951eb77-openshift-default-85859ff58c-gf998 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9 to ip-10-0-134-253.us-west-1.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-994b7

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-994b7 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm to ip-10-0-129-92.us-west-1.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-bpkrz

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-bpkrz to ip-10-0-129-92.us-west-1.compute.internal

llm

e2e-same-id-b-f6cbc132

ModelNameConflictResolved

Model identity is no longer shared with any other MaaSModelRef in this namespace

llm

e2e-same-id-b-f6cbc132

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-a-f6cbc132

openshift-ingress

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw to ip-10-0-141-97.us-west-1.compute.internal

openshift-jobset-operator

jobset-controller-manager-956f9bf55-95jwv

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-956f9bf55-95jwv to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb to ip-10-0-129-92.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-xrxx5 to ip-10-0-129-92.us-west-1.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9 to ip-10-0-141-97.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-vkmcw to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g to ip-10-0-141-97.us-west-1.compute.internal

kuadrant-system

authorino-5584c7d87f-wjpw2

Scheduled

Successfully assigned kuadrant-system/authorino-5584c7d87f-wjpw2 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn to ip-10-0-134-253.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-85d75859fc-5shzd

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-85d75859fc-5shzd to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-cdt6d to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd to ip-10-0-129-92.us-west-1.compute.internal

opendatahub

test-curl-7446-a58b91

Scheduled

Successfully assigned opendatahub/test-curl-7446-a58b91 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

Scheduled

Successfully assigned openshift-ingress/e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gwt5l to ip-10-0-141-97.us-west-1.compute.internal

kuadrant-system

authorino-56c6c9ddd5-6xsvt

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-6xsvt to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

test-curl-7446-9f4e17

Scheduled

Successfully assigned opendatahub/test-curl-7446-9f4e17 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp to ip-10-0-141-97.us-west-1.compute.internal

llm

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

Scheduled

Successfully assigned llm/e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8 to ip-10-0-129-92.us-west-1.compute.internal

llm

e2e-same-id-a-f6cbc132

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-b-f6cbc132

openshift-ingress

payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7 to ip-10-0-129-92.us-west-1.compute.internal

opendatahub

test-curl-7443-a7f194

Scheduled

Successfully assigned opendatahub/test-curl-7443-a7f194 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8 to ip-10-0-129-92.us-west-1.compute.internal

kuadrant-system

authorino-57f7f8659f-tzz7h

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-tzz7h to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-noten-40af1b-openshift-default-79966c65b5-lt94f

Scheduled

Successfully assigned openshift-ingress/e2e-noten-40af1b-openshift-default-79966c65b5-lt94f to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

test-curl-7443-5cc14f

Scheduled

Successfully assigned opendatahub/test-curl-7443-5cc14f to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-8f57g

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-8f57g to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-faa8ff-openshift-default-667f695f57-982wc to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-lpklt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-lpklt to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z to ip-10-0-141-97.us-west-1.compute.internal

kuadrant-system

authorino-6547cd65c-7v7ck

Scheduled

Successfully assigned kuadrant-system/authorino-6547cd65c-7v7ck to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

test-curl-7443-2bc023

Scheduled

Successfully assigned opendatahub/test-curl-7443-2bc023 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-5fb6d674c6-8c4kb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-5fb6d674c6-8c4kb to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-j678p

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-j678p to ip-10-0-141-97.us-west-1.compute.internal

kuadrant-system

authorino-6f6944bcdd-b89n4

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-b89n4 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

test-curl-7443-0b0e22

Scheduled

Successfully assigned opendatahub/test-curl-7443-0b0e22 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

Scheduled

Successfully assigned openshift-ingress/e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-4jxzw to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

test-curl-7440-fed722

Scheduled

Successfully assigned opendatahub/test-curl-7440-fed722 to ip-10-0-134-253.us-west-1.compute.internal

kuadrant-system

authorino-847cf6dc9d-lk8pc

Scheduled

Successfully assigned kuadrant-system/authorino-847cf6dc9d-lk8pc to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

Scheduled

Successfully assigned openshift-ingress/e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-55775b4dc9-fvzqg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-55775b4dc9-fvzqg to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-wl8rv to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv to ip-10-0-134-253.us-west-1.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-k2nkj

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-k2nkj to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

test-curl-7440-705aa1

Scheduled

Successfully assigned opendatahub/test-curl-7440-705aa1 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-mpdzz

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-mpdzz to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-p22bd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-p22bd to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-px79s

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-px79s to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-vf6t5 to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

test-curl-7440-3ab145

Scheduled

Successfully assigned opendatahub/test-curl-7440-3ab145 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-trkbg to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh to ip-10-0-129-92.us-west-1.compute.internal

opendatahub

test-curl-7440-1d3127

Scheduled

Successfully assigned opendatahub/test-curl-7440-1d3127 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-b6ff555c6-lxd69

Scheduled

Successfully assigned openshift-ingress/payload-processing-b6ff555c6-lxd69 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-6dbdbcbf6f-6dq5z

Scheduled

Successfully assigned openshift-ingress/payload-processing-6dbdbcbf6f-6dq5z to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

test-curl-7440-108a1d

Scheduled

Successfully assigned opendatahub/test-curl-7440-108a1d to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

odh-model-controller-769875db56-pjms7

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-pjms7 to ip-10-0-129-92.us-west-1.compute.internal

opendatahub

model-serving-api-75bd89fdfc-h2pkm

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-h2pkm to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-4s7z6

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-4s7z6 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-5ljv7

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-5ljv7 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-kdh6t

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-kdh6t to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5 to ip-10-0-129-92.us-west-1.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-9t2hp

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-9t2hp to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-7g89g

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-7g89g to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5 to ip-10-0-129-92.us-west-1.compute.internal

opendatahub

maas-controller-74b5bc6fff-vqr9s

Scheduled

Successfully assigned opendatahub/maas-controller-74b5bc6fff-vqr9s to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-55fbbbf9dc-rfb94

Scheduled

Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-rfb94 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

Scheduled

Successfully assigned openshift-ingress/e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

maas-controller-74b5bc6fff-gmvbm

Scheduled

Successfully assigned opendatahub/maas-controller-74b5bc6fff-gmvbm to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8lzd5 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

maas-controller-74b5bc6fff-6xcrb

Scheduled

Successfully assigned opendatahub/maas-controller-74b5bc6fff-6xcrb to ip-10-0-129-92.us-west-1.compute.internal

opendatahub

maas-controller-6f8cfc466-hdgmh

Scheduled

Successfully assigned opendatahub/maas-controller-6f8cfc466-hdgmh to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-4mqpn

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-4mqpn to ip-10-0-134-253.us-west-1.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-crc2n

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-crc2n to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-8krz2

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-8krz2 to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-8qncv

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-8qncv to ip-10-0-134-253.us-west-1.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-hstph

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-hstph to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

ai-gateway-controller-7d85675f98-ffpn8

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-7d85675f98-ffpn8 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-sgdbb to ip-10-0-129-92.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-kfkl9

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-kfkl9 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-r5dmv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-r5dmv to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7 to ip-10-0-141-97.us-west-1.compute.internal

opendatahub

ai-gateway-controller-559cb95fbf-czdwv

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-559cb95fbf-czdwv to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-sd9zb

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-sd9zb to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-8ng9q

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-8ng9q to ip-10-0-141-97.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-rcf8t

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-rcf8t to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-kb7j2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-kb7j2 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-qpjb7 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5 to ip-10-0-141-97.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-9rwv6

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9rwv6 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-p6p56 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-tgfm2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-tgfm2 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-tmwv2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-tmwv2 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-nkc44 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-h5t46

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-h5t46 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-94m8z

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-94m8z to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-m5kl4 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-kf57z to ip-10-0-129-92.us-west-1.compute.internal

ai-tenants

e2e-autores-ad05c6

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-autores-ad05c6: MaasTenantConfig is not available to restore revocation operands

kuadrant-system

limitador-limitador-74dfcfb894-9ljw8

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-9ljw8 to ip-10-0-141-97.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-ff8dd698-dfp4l

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-ff8dd698-dfp4l to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-9gnnl

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-9gnnl to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-b4bkr

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-b4bkr to ip-10-0-141-97.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-ff8dd698-cvrtg

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-ff8dd698-cvrtg to ip-10-0-141-97.us-west-1.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-dc89h

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-dc89h to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-gxmrv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-gxmrv to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j8mfj to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-j7t9j to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-c2zj8 to ip-10-0-129-92.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-848b684699-zqxjt

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-848b684699-zqxjt to ip-10-0-129-92.us-west-1.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-wttv8

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-wttv8 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-f24gh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-f24gh to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28 to ip-10-0-141-97.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-6c9f596676-2bds7

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6c9f596676-2bds7 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-c6zlb

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-c6zlb to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl to ip-10-0-141-97.us-west-1.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-jd2fn

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-jd2fn to ip-10-0-129-92.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-67d67f47f4-bp4gq

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-67d67f47f4-bp4gq to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-gtrfd to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj to ip-10-0-141-97.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-5bcd7997-knxw6

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5bcd7997-knxw6 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-gk2pd to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-cd9pn

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-cd9pn to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-vwd6x to ip-10-0-129-92.us-west-1.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll to ip-10-0-129-92.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-59599f94ff-dzx6z

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-59599f94ff-dzx6z to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-fchpl to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-w2nmc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-w2nmc to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-xnxx8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xnxx8 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-xsdd7

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-xsdd7 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-cdk5z to ip-10-0-141-97.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-59599f94ff-6qzvs

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-59599f94ff-6qzvs to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-7gfxb to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-z24p2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-z24p2 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-6x4ct to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-5bms2 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

Scheduled

Successfully assigned openshift-ingress/e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g to ip-10-0-134-253.us-west-1.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-mf4nj

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-mf4nj to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw to ip-10-0-129-92.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-57896c8f89-hx4zg

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-57896c8f89-hx4zg to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

router-default-74dd76c985-n79rt

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-74dd76c985-n79rt

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-74dd76c985-n79rt

Scheduled

Successfully assigned openshift-ingress/router-default-74dd76c985-n79rt to ip-10-0-134-253.us-west-1.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-qbk8d

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-qbk8d to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

Scheduled

Successfully assigned openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-v8m4h to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t to ip-10-0-141-97.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-5565547b4b-2kj8t

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5565547b4b-2kj8t to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-a48099-openshift-default-577864759-4rqx7 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7 to ip-10-0-141-97.us-west-1.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-d797t

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-d797t to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-zml98

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-zml98 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-ck46t

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ck46t to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-mt-25565930-openshift-default-64f5666649-5crwq

Scheduled

Successfully assigned openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-5crwq to ip-10-0-141-97.us-west-1.compute.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz to ip-10-0-141-97.us-west-1.compute.internal

llm

e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh

Scheduled

Successfully assigned llm/e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x to ip-10-0-141-97.us-west-1.compute.internal

llm

e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8

Scheduled

Successfully assigned llm/e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8 to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-zzhvc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-zzhvc to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

Scheduled

Successfully assigned openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-cpzlj to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw to ip-10-0-129-92.us-west-1.compute.internal

llm

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

Scheduled

Successfully assigned llm/e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

Scheduled

Successfully assigned openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-gbhn9 to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

Scheduled

Successfully assigned openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-gmkbz to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr to ip-10-0-129-92.us-west-1.compute.internal

openshift-ingress

e2e-mt-25565930-openshift-default-64f5666649-rq68b

Scheduled

Successfully assigned openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-rq68b to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-zpxqq

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-zpxqq to ip-10-0-129-92.us-west-1.compute.internal

ai-tenants

e2e-api-b-8815d2

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-b-8815d2: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc to ip-10-0-141-97.us-west-1.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

Scheduled

Successfully assigned openshift-ingress/e2e-mt-25565930-openshift-default-64f5666649-wjfgh to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8 to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj to ip-10-0-134-253.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-58d898bd55-ztp4p

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-58d898bd55-ztp4p to ip-10-0-141-97.us-west-1.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-74dd76c985 from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-74dd76c985

SuccessfulCreate

Created pod: router-default-74dd76c985-n79rt
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-74dd76c985-n79rt

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-74dd76c985-n79rt

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x3)

openshift-ingress

kubelet

router-default-74dd76c985-n79rt

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x3)

openshift-ingress

kubelet

router-default-74dd76c985-n79rt

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-74dd76c985-n79rt

Created

Container created

openshift-ingress

kubelet

router-default-74dd76c985-n79rt

Started

Container started

openshift-ingress

kubelet

router-default-74dd76c985-n79rt

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

multus

router-default-74dd76c985-n79rt

AddedInterface

Add eth0 [10.132.0.14/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-8f57g

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-8f57g

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-8f57g

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-8f57g

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-8f57g

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.979s (3.979s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-8f57g

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-8f57g

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-8f57g

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-58d898bd55 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-ck46t

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-649845f967

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-649845f967 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-ck46t

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 11.29s (11.29s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v

Created

Container created

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-649845f967-d9n5v

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.794s (10.794s including waiting). Image size: 183555333 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-85d75859fc

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-85d75859fc-vkmcw

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-85d75859fc

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-85d75859fc-xrxx5

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-85d75859fc from 0 to 3

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-85d75859fc

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-85d75859fc-5shzd

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-85d75859fc-vkmcw

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-85d75859fc-5shzd

AddedInterface

Add eth0 [10.133.0.18/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-85d75859fc-xrxx5

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-qbk8d

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.405s (3.405s including waiting). Image size: 273709574 bytes.
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-qbk8d

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-qbk8d

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.298s (4.298s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 772ms (772ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.286s (4.286s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-xrxx5

Started

Container started
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 798ms (798ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-vkmcw

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-qbk8d

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-qbk8d

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Started

Container started
(x2)

opendatahub-operator-system

operator-lifecycle-manager

install-rmvxf

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1

opendatahub-operator-system

opendatahub-operator-controller-manager-85d75859fc-xrxx5_959067d8-40cd-4a27-8eb0-285207201318

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-85d75859fc-xrxx5_959067d8-40cd-4a27-8eb0-285207201318 became leader

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-qbk8d

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 2.917s (2.917s including waiting). Image size: 224874686 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-85d75859fc-5shzd

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 859ms (860ms including waiting). Image size: 273709574 bytes.

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-qbk8d_869d64f3-4938-4176-9f13-600a9207ca48 became leader

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-k2nkj

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-k2nkj

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

kuadrant-system

multus

authorino-operator-7b7bb4bc77-k2nkj

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1
(x6)

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

FailedCreate

Error creating: pods "cert-manager-webhook-5dbbbf75cc-" is forbidden: error looking up service account cert-manager/cert-manager-webhook: serviceaccount "cert-manager-webhook" not found

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-994b7

AddedInterface

Add eth0 [10.133.0.19/23] from ovn-kubernetes

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-994b7
(x9)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-994b7

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-kc9p7

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-kc9p7

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-kc9p7

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-k2nkj

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 5.962s (5.962s including waiting). Image size: 167738465 bytes.

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-k2nkj

Started

Container started

kuadrant-system

authorino-operator-7b7bb4bc77-k2nkj_ebe0d006-44f3-4401-8819-426c191e49bc

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-k2nkj_ebe0d006-44f3-4401-8819-426c191e49bc became leader

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-9rwv6

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-k2nkj

Created

Container created

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-9rwv6

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9rwv6

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-kc9p7

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-994b7

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-kc9p7

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.738s (3.738s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-994b7

Created

Container created

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-994b7

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 5.406s (5.406s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-kc9p7

Created

Container created

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-wttv8

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-wttv8

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-wttv8

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes
(x12)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9rwv6

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9rwv6

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 5.348s (5.348s including waiting). Image size: 424244516 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9rwv6

Started

Container started

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-wttv8

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 2.538s (2.538s including waiting). Image size: 177967631 bytes.

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-9rwv6_0db010b2-d5d7-4a6b-b1a5-296a66bee2ee

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-9rwv6_0db010b2-d5d7-4a6b-b1a5-296a66bee2ee became leader

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-wttv8

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-wttv8

Started

Container started

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

kuadrant-system

limitador-operator-controller-manager-5778bb4449-wttv8_c48524ee-9397-4589-9b5a-1bbb37d7d0e5

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-wttv8_c48524ee-9397-4589-9b5a-1bbb37d7d0e5 became leader

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1
(x3)

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreateFailed

Failed to create Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator: Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": no endpoints available for service "cert-manager-webhook"

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-9t2hp

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-9t2hp

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-9t2hp

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-5f68b7bd84 from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-5f68b7bd84

SuccessfulCreate

Created pod: kube-auth-proxy-5f68b7bd84-5rtgt

openshift-ingress

replicaset-controller

kube-auth-proxy-5f68b7bd84

SuccessfulCreate

Created pod: kube-auth-proxy-5f68b7bd84-nfg9z

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-69f549cf5c from 0 to 1

openshift-ingress

multus

kube-auth-proxy-5f68b7bd84-5rtgt

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-5rtgt

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-5f68b7bd84-nfg9z

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-69f549cf5c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-nfg9z

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9

Started

Container started

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-69f549cf5cfwfn9

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-9t2hp

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 2.623s (2.623s including waiting). Image size: 190804149 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-9t2hp

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-9t2hp

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-nfg9z

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.763s (2.763s including waiting). Image size: 188904677 bytes.

kuadrant-system

dns-operator-controller-manager-8557cbdb84-9t2hp_b8d43c4d-b5f3-4ba9-a8f3-e8e5beb5f97d

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-9t2hp_b8d43c4d-b5f3-4ba9-a8f3-e8e5beb5f97d became leader

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-nfg9z

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-5rtgt

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.808s (2.808s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-5rtgt

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-5rtgt

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-5f68b7bd84-nfg9z

Created

Container created

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-fztvp

cert-manager

kubelet

cert-manager-597b95cb84-fztvp

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

cert-manager

multus

cert-manager-597b95cb84-fztvp

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-fztvp

Created

Container created

cert-manager

kubelet

cert-manager-597b95cb84-fztvp

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-kfkl9

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-kfkl9

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-kfkl9

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-kfkl9

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-kfkl9

Started

Container started

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

kuadrant-system

multus

limitador-limitador-74dfcfb894-9ljw8

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9ljw8

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-9ljw8

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-tzz7h

kuadrant-system

multus

authorino-57f7f8659f-tzz7h

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-57f7f8659f-tzz7h

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9ljw8

Started

Container started

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9ljw8

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9ljw8

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 3.455s (3.455s including waiting). Image size: 133379307 bytes.

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-dc89h

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-6xsvt

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

multus

limitador-limitador-7c9b87d574-dc89h

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

kuadrant-system

multus

authorino-56c6c9ddd5-6xsvt

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-56c6c9ddd5-6xsvt

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-dc89h

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-9rwv6

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-dc89h

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 1.691s (1.691s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-dc89h

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-dc89h

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-tzz7h

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-tzz7h

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9rwv6

Killing

Stopping container manager

kuadrant-system

kubelet

authorino-57f7f8659f-tzz7h

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.67s (3.67s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-6xsvt

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-tzz7h

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-56c6c9ddd5-6xsvt

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.945s (2.945s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-6xsvt

Created

Container created

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-tzz7h

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-8qncv

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-8qncv

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-8qncv

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-8qncv

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-8qncv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 3.216s (3.216s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-8qncv

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9ljw8

Killing

Stopping container limitador

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-9ljw8

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-skp8k"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-lgj5q"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-956f9bf55 from 0 to 1

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-956f9bf55

SuccessfulCreate

Created pod: jobset-controller-manager-956f9bf55-95jwv

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-jobset-operator

kubelet

jobset-controller-manager-956f9bf55-95jwv

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

openshift-jobset-operator

multus

jobset-controller-manager-956f9bf55-95jwv

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-kfkl9_975149d4-8b19-4dab-ae5d-cc27ae061206

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-kfkl9_975149d4-8b19-4dab-ae5d-cc27ae061206 became leader

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

openshift-jobset-operator

kubelet

jobset-controller-manager-956f9bf55-95jwv

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 4.115s (4.115s including waiting). Image size: 182184678 bytes.

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-crc2n

openshift-jobset-operator

kubelet

jobset-controller-manager-956f9bf55-95jwv

Started

Container started

openshift-jobset-operator

kubelet

jobset-controller-manager-956f9bf55-95jwv

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-crc2n

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-crc2n

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-jobset-operator

jobset-controller-manager-956f9bf55-95jwv_0ce8de2e-6439-48a1-80cc-5fe2d57f7bdb

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-956f9bf55-95jwv_0ce8de2e-6439-48a1-80cc-5fe2d57f7bdb became leader

opendatahub

kserve-module-controller-manager-7566cf5fc8-8qncv_e6adf38f-0473-4159-9b1b-ccfe6fa95e10

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-8qncv_e6adf38f-0473-4159-9b1b-ccfe6fa95e10 became leader

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-crc2n

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 5.646s (5.646s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-crc2n

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-crc2n

Started

Container started

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-6f8cfc466 from 0 to 1

opendatahub

replicaset-controller

maas-controller-6f8cfc466

SuccessfulCreate

Created pod: maas-controller-6f8cfc466-hdgmh

opendatahub

replicaset-controller

maas-controller-74b5bc6fff

SuccessfulCreate

Created pod: maas-controller-74b5bc6fff-vqr9s

opendatahub

multus

maas-controller-6f8cfc466-hdgmh

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-6f8cfc466-hdgmh

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19"

opendatahub

multus

maas-controller-74b5bc6fff-vqr9s

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-6f8cfc466-hdgmh

Started

Container started

opendatahub

kubelet

maas-controller-6f8cfc466-hdgmh

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19" in 6.543s (6.543s including waiting). Image size: 200094741 bytes.

opendatahub

kubelet

maas-controller-74b5bc6fff-vqr9s

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19" in 5.988s (5.988s including waiting). Image size: 200094741 bytes.

opendatahub

kubelet

maas-controller-6f8cfc466-hdgmh

Created

Container created

opendatahub

maas-controller-6f8cfc466-hdgmh_a6308314-06c2-48f3-98c6-bf492a370456

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-6f8cfc466-hdgmh_a6308314-06c2-48f3-98c6-bf492a370456 became leader

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

FailedMount

MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-6f8cfc466 from 1 to 0

opendatahub

kubelet

maas-controller-6f8cfc466-hdgmh

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-6f8cfc466

SuccessfulDelete

Deleted pod: maas-controller-6f8cfc466-hdgmh

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-zpxqq

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-zpxqq

Started

Container started

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-zpxqq

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-zpxqq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-zpxqq

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-zpxqq

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-zpxqq

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-8f57g

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-8f57g

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.038s (13.038s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-cdt6d

Created

Container created

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-4mqpn

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-pjms7

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-h2pkm

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-hstph

opendatahub

kubelet

odh-model-controller-769875db56-pjms7

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

multus

model-serving-api-75bd89fdfc-h2pkm

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

opendatahub

multus

kserve-controller-manager-776cc66d47-hstph

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

opendatahub

kubelet

kserve-controller-manager-776cc66d47-hstph

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-4mqpn

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-4mqpn

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-h2pkm

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

multus

odh-model-controller-769875db56-pjms7

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-h2pkm

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-h2pkm

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.295s (2.295s including waiting). Image size: 184925761 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-h2pkm

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-4mqpn

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.359s (2.359s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-4mqpn

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-4mqpn

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kubelet

kserve-controller-manager-776cc66d47-hstph

Started

Container started

opendatahub

kubelet

kserve-controller-manager-776cc66d47-hstph

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-hstph

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.781s (2.781s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-pjms7

Created

Container created

opendatahub

kserve-controller-manager-776cc66d47-hstph_55448ed4-1fce-44bf-8fe4-6ab35e9d8b1e

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-hstph_55448ed4-1fce-44bf-8fe4-6ab35e9d8b1e became leader

opendatahub

llmisvc-controller-manager-869d5dcc5d-4mqpn_a4f692e1-e37b-4cc6-ba5e-e2248445c7f3

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-4mqpn_a4f692e1-e37b-4cc6-ba5e-e2248445c7f3 became leader

opendatahub

kubelet

odh-model-controller-769875db56-pjms7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 4.368s (4.368s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-pjms7

Started

Container started

opendatahub

odh-model-controller-769875db56-pjms7_7a9f90c4-cf98-48f4-b3b8-51b0cf4f20a6

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-pjms7_7a9f90c4-cf98-48f4-b3b8-51b0cf4f20a6 became leader

opendatahub

maas-controller-74b5bc6fff-vqr9s_cc10d712-8fff-43fb-8905-08dc991901ae

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-74b5bc6fff-vqr9s_cc10d712-8fff-43fb-8905-08dc991901ae became leader
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-b89n4

kuadrant-system

kubelet

authorino-6f6944bcdd-b89n4

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-5584c7d87f-wjpw2

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6547cd65c from 0 to 1

kuadrant-system

kubelet

authorino-5584c7d87f-wjpw2

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-6f6944bcdd-b89n4

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-6547cd65c

SuccessfulCreate

Created pod: authorino-6547cd65c-7v7ck

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-b89n4

kuadrant-system

replicaset-controller

authorino-5584c7d87f

SuccessfulCreate

Created pod: authorino-5584c7d87f-wjpw2

kuadrant-system

replicaset-controller

authorino-5584c7d87f

SuccessfulDelete

Deleted pod: authorino-5584c7d87f-wjpw2

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-5584c7d87f from 1 to 0

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-5584c7d87f from 0 to 1

kuadrant-system

kubelet

authorino-6f6944bcdd-b89n4

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.444s (1.444s including waiting). Image size: 204096810 bytes.

kuadrant-system

multus

authorino-6547cd65c-7v7ck

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-6547cd65c-7v7ck

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-6f6944bcdd-b89n4

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-b89n4

Started

Container started

kuadrant-system

kubelet

authorino-6f6944bcdd-b89n4

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-6547cd65c-7v7ck

Started

Container started

kuadrant-system

kubelet

authorino-6547cd65c-7v7ck

Created

Container created

kuadrant-system

kubelet

authorino-5584c7d87f-wjpw2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.263s (1.263s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-5584c7d87f-wjpw2

Created

Container created

kuadrant-system

kubelet

authorino-5584c7d87f-wjpw2

Started

Container started

kuadrant-system

kubelet

authorino-6547cd65c-7v7ck

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.187s (1.187s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-5584c7d87f-wjpw2

Killing

Stopping container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

kubelet

authorino-56c6c9ddd5-6xsvt

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-6xsvt

opendatahub

multus

llmisvc-controller-manager-6466c554d8-8krz2

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-8krz2

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-8krz2

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-8krz2

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 739ms (739ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-8krz2

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-8krz2

Started

Container started

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-4mqpn

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-4mqpn

Killing

Stopping container manager

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-7d85675f98 from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-7d85675f98

SuccessfulCreate

Created pod: ai-gateway-controller-7d85675f98-ffpn8

opendatahub

multus

ai-gateway-controller-7d85675f98-ffpn8

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-7d85675f98-ffpn8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-7d85675f98-ffpn8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 2.855s (2.855s including waiting). Image size: 183979433 bytes.

opendatahub

kubelet

ai-gateway-controller-7d85675f98-ffpn8

Started

Container started

opendatahub

kubelet

ai-gateway-controller-7d85675f98-ffpn8

Created

Container created

opendatahub

ai-gateway-controller-7d85675f98-ffpn8_0b6ea0d0-5c11-4b0b-9503-29c37d9280e0

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-7d85675f98-ffpn8_0b6ea0d0-5c11-4b0b-9503-29c37d9280e0 became leader

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulCreate

Created pod: payload-processing-55fbbbf9dc-rfb94
(x7)

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

FailedCreate

Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-j678p

openshift-ingress

multus

payload-pre-processing-7b98bb678c-j678p

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-55fbbbf9dc-rfb94

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j678p

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rfb94

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j678p

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j678p

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j678p

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.769s (2.769s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rfb94

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.713s (2.713s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rfb94

Created

Container created

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rfb94

Started

Container started

opendatahub

llmisvc-controller-manager-6466c554d8-8krz2_c891e6b6-951e-43fe-adf5-330aa2a393f9

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-8krz2_c891e6b6-951e-43fe-adf5-330aa2a393f9 became leader

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-559cb95fbf from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-559cb95fbf

SuccessfulCreate

Created pod: ai-gateway-controller-559cb95fbf-czdwv

opendatahub

multus

ai-gateway-controller-559cb95fbf-czdwv

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

Created

Container created
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

kubelet

ai-gateway-controller-559cb95fbf-czdwv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 787ms (787ms including waiting). Image size: 183979433 bytes.
(x2)

opendatahub

kubelet

maas-controller-74b5bc6fff-vqr9s

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19"
(x2)

opendatahub

kubelet

maas-controller-74b5bc6fff-vqr9s

Created

Container created
(x2)

opendatahub

kubelet

maas-controller-74b5bc6fff-vqr9s

Started

Container started

opendatahub

kubelet

maas-controller-74b5bc6fff-vqr9s

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19" in 917ms (917ms including waiting). Image size: 200094741 bytes.
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

Unhealthy

Startup probe failed: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

ProbeError

Startup probe error: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused body:

opendatahub

replicaset-controller

ai-gateway-controller-7d85675f98

SuccessfulDelete

Deleted pod: ai-gateway-controller-7d85675f98-ffpn8

opendatahub

kubelet

ai-gateway-controller-7d85675f98-ffpn8

Killing

Stopping container manager

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-7d85675f98 from 1 to 0

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

opendatahub

maas-controller-74b5bc6fff-vqr9s_33a2797c-4b9f-406e-a854-3d232f5ffd62

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-74b5bc6fff-vqr9s_33a2797c-4b9f-406e-a854-3d232f5ffd62 became leader

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.217s (2.217s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-6h4ll

Created

Container created

opendatahub

ai-gateway-controller-559cb95fbf-czdwv_cbe25692-e5c2-40d3-bc07-2b8e94c64ff5

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-559cb95fbf-czdwv_cbe25692-e5c2-40d3-bc07-2b8e94c64ff5 became leader

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-mf4nj

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-mf4nj

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-mf4nj

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-mf4nj

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.7s (1.7s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-mf4nj

Created

Container created

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-mf4nj

Started

Container started

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-d797t

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-d797t

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-d797t

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 598ms (598ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-d797t

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-d797t

Created

Container created

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-tn2cx

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-7g89g

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-4jxzw

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-d797t

Started

Container started

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-tn2cx

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-4jxzw

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-7g89g

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-bpkrz

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-bpkrz

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-bpkrz

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-bpkrz

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 575ms (575ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-bpkrz

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-bpkrz

Started

Container started

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.939s (3.939s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.695s (4.695s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.3s (4.3s including waiting). Image size: 183555333 bytes.

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Started

Container started

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 723ms (723ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8mgng5

Started

Container started

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-jd2fn

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

kubelet

echo-simulator-kserve-64b6f8f96c-jd2fn

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

multus

echo-simulator-kserve-64b6f8f96c-jd2fn

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

Unhealthy

Startup probe failed: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

ProbeError

Startup probe error: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Unhealthy

Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused

llm

kubelet

echo-simulator-kserve-64b6f8f96c-jd2fn

Started

Container started
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Unhealthy

Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

ProbeError

Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

ProbeError

Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused body:

llm

kubelet

echo-simulator-kserve-64b6f8f96c-jd2fn

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.579s (2.579s including waiting). Image size: 78208878 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-jd2fn

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Unhealthy

Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

ProbeError

Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Unhealthy

Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

ProbeError

Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason:

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-lpklt

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-94m8z

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-8lzd5

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-2jwpv

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-lpklt

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

Started

Container started

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 716ms (716ms including waiting). Image size: 98346788 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-2jwpv

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rztr

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-8lzd5

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-94m8z

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

Created

Container created

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

Unhealthy

Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

Started

Container started
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

ProbeError

Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

Unhealthy

Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 810ms (810ms including waiting). Image size: 98346788 bytes.
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

ProbeError

Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

ProbeError

Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Unhealthy

Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

ProbeError

Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

Unhealthy

Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

ProbeError

Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

Unhealthy

Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Unhealthy

Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

ProbeError

Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-58d898bd55-f24gh

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-h5t46

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-h5t46

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-f24gh

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-h5t46

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-f24gh

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-f24gh

Created

Container created

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-h5t46

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-h5t46

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-f24gh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-f24gh

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-f24gh

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-h5t46

Unhealthy

Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-h5t46

ProbeError

Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused body:

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

kuadrant-system

replicaset-controller

authorino-847cf6dc9d

SuccessfulCreate

Created pod: authorino-847cf6dc9d-lk8pc

kuadrant-system

kubelet

authorino-847cf6dc9d-lk8pc

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-847cf6dc9d-lk8pc

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-847cf6dc9d-lk8pc

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.714s (1.714s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-847cf6dc9d-lk8pc

Started

Container started

kuadrant-system

kubelet

authorino-847cf6dc9d-lk8pc

Created

Container created
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-6547cd65c from 1 to 0

kuadrant-system

kubelet

authorino-6547cd65c-7v7ck

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-6547cd65c

SuccessfulDelete

Deleted pod: authorino-6547cd65c-7v7ck

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-3d70ff

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-3d70ff

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-7d1ec3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-7d1ec3

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-a48099

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-a48099

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 0 to 1

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-a48099

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-a48099-6566d45677 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-a48099: serviceaccount "payload-processing-e2e-worker-w4-a48099" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-a48099-6566d45677

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-a48099-6566d45677-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-a48099: serviceaccount "payload-processing-e2e-worker-w4-a48099" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-a48099

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-a48099-786fbc588c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-a48099-6566d45677

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-7d1ec3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-7d1ec3: serviceaccount "payload-processing-e2e-worker-w0-7d1ec3" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-7d1ec3: serviceaccount "payload-processing-e2e-worker-w0-7d1ec3" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-7d1ec3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

Started

Container started
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-3d70ff: serviceaccount "payload-processing-e2e-worker-w2-3d70ff" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-3d70ff

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-3d70ff-6dc999899 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-3d70ff-6dc999899

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-3d70ff-6dc999899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-3d70ff: serviceaccount "payload-processing-e2e-worker-w2-3d70ff" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-3d70ff

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-3d70ff-6dc999899

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

Unhealthy

Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

ProbeError

Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

ProbeError

Readiness probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

Unhealthy

Readiness probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

Unhealthy

Readiness probe failed: Get "http://10.134.0.43:15021/healthz/ready": read tcp 10.134.0.2:34502->10.134.0.43:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

ProbeError

Readiness probe error: Get "http://10.134.0.43:15021/healthz/ready": read tcp 10.134.0.2:34502->10.134.0.43:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

ProbeError

Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

Unhealthy

Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

ProbeError

Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

Unhealthy

Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-inf-a-2ea125-openshift-default-dd6f78f5f

SuccessfulCreate

Created pod: e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

openshift-ingress

deployment-controller

e2e-inf-a-2ea125-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-2ea125-openshift-default-dd6f78f5f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-a-2ea125

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-2ea125

openshift-ingress

multus

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-2ea125-5cc577d6b5

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-2ea125-5cc577d6b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2ea125: serviceaccount "payload-processing-e2e-inf-a-2ea125" not found

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-2ea125

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-2ea125-5cc577d6b5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-2ea125

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-2ea125-595686979 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-2ea125-5cc577d6b5

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

openshift-ingress

multus

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-2ea125-595686979

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-2ea125-595686979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2ea125: serviceaccount "payload-processing-e2e-inf-a-2ea125" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-2ea125-595686979

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.662s (2.662s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Started

Container started
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

ProbeError

Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

Unhealthy

Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-inf-b-af8051

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-af8051

openshift-ingress

multus

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-inf-b-af8051-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-af8051-openshift-default-6cb5867fc5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-b-af8051-openshift-default-6cb5867fc5

SuccessfulCreate

Created pod: e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-2ea125-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-2ea125-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-af8051

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-af8051-5599f954b9 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-af8051-79d78db899

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-af8051-79d78db899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-af8051: serviceaccount "payload-processing-e2e-inf-b-af8051" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-af8051-5599f954b9

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-af8051

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-af8051-79d78db899 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-af8051-79d78db899

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-af8051-5599f954b9

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-af8051-5599f954b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-af8051: serviceaccount "payload-processing-e2e-inf-b-af8051" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-2ea125-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-2ea125-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

test-curl-7443-a7f194

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7443-a7f194

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7443-a7f194

Started

Container started

opendatahub

kubelet

test-curl-7443-a7f194

Created

Container created

opendatahub

kubelet

test-curl-7443-a7f194

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.152s (3.152s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7443-2bc023

Started

Container started

opendatahub

kubelet

test-curl-7443-2bc023

Created

Container created

opendatahub

kubelet

test-curl-7443-2bc023

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7443-a7f194

Killing

Stopping container test-curl-7443-a7f194

opendatahub

multus

test-curl-7443-2bc023

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7443-2bc023

Killing

Stopping container test-curl-7443-2bc023

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-a12c35

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-a12c35

openshift-ingress

multus

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-a-a12c35-openshift-default-84d7454997

SuccessfulCreate

Created pod: e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

openshift-ingress

deployment-controller

e2e-extmod-a-a12c35-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-a12c35-openshift-default-84d7454997 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-a12c35

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-a12c35-6cdb45475c from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-a12c35-6cdb45475c

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-a12c35-6cdb45475c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-a12c35: serviceaccount "payload-processing-e2e-extmod-a-a12c35" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-a12c35

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-a12c35: serviceaccount "payload-processing-e2e-extmod-a-a12c35" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-a12c35-6cdb45475c

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

Started

Container started

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

Unhealthy

Readiness probe failed: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

ProbeError

Readiness probe error: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

Unhealthy

Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

ProbeError

Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-af8051-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-af8051-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-af8051-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-af8051-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

ProbeError

Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

Unhealthy

Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a12c35-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a12c35-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 1 to 2

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

Started

Container started

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-2fd4c9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-2fd4c9

openshift-ingress

deployment-controller

e2e-extmod-b-2fd4c9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f from 0 to 1

openshift-ingress

multus

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f

SuccessfulCreate

Created pod: e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

Unhealthy

Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

ProbeError

Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-2fd4c9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-2fd4c9-7b49b85457 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-2fd4c9: serviceaccount "payload-processing-e2e-extmod-b-2fd4c9" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-2fd4c9: serviceaccount "payload-processing-e2e-extmod-b-2fd4c9" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-2fd4c9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

openshift-ingress

multus

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a12c35-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-a12c35-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

Created

Container created
(x7)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

Unhealthy

Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

ProbeError

Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2fd4c9-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2fd4c9-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-2864b6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-2864b6

openshift-ingress

deployment-controller

e2e-ipp-a-2864b6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-2864b6-openshift-default-864f5c8fbc from 0 to 1

openshift-ingress

multus

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc

SuccessfulCreate

Created pod: e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 2 to 3

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

Started

Container started

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

Created

Container created

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-p6p56

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2864b6: serviceaccount "payload-processing-e2e-ipp-a-2864b6" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-2864b6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2864b6-768dff758f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-2864b6-768dff758f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2864b6: serviceaccount "payload-processing-e2e-ipp-a-2864b6" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-2864b6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-2864b6-768dff758f from 0 to 1
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2864b6-768dff758f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw

openshift-ingress

multus

payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

Unhealthy

Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

ProbeError

Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2fd4c9-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 3 to 4
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2fd4c9-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

AddedInterface

Add eth0 [10.133.0.81/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

Unhealthy

Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

ProbeError

Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

ProbeError

Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

Unhealthy

Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-h5t46

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-4jxzw

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-2jwpv

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-f24gh

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-lpklt

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-58d898bd55 from 10 to 5

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-h5t46

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-lpklt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4jxzw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-2jwpv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-f24gh

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 4 to 6

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

Started

Container started

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

Unhealthy

Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

ProbeError

Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

Unhealthy

Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

ProbeError

Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

Started

Container started

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

AddedInterface

Add eth0 [10.133.0.88/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

Unhealthy

Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

ProbeError

Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

Unhealthy

Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

ProbeError

Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

Unhealthy

Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

ProbeError

Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

ProbeError

Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

Unhealthy

Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 5 to 7

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-kf57z

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-5451cb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-5451cb

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

Started

Container started

openshift-ingress

deployment-controller

e2e-ipp-b-5451cb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-5451cb-openshift-default-5965d59997 from 0 to 1

openshift-ingress

multus

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ipp-b-5451cb-openshift-default-5965d59997

SuccessfulCreate

Created pod: e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-768dff758f-r9m4h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-d5585ff4c-x2hfw

Killing

Stopping container payload-pre-processing
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-2864b6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-2864b6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-5451cb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-5451cb-84846d58f9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-5451cb-84846d58f9

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-5451cb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5451cb: serviceaccount "payload-processing-e2e-ipp-b-5451cb" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-5451cb-84846d58f9

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-5451cb-84846d58f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5451cb: serviceaccount "payload-processing-e2e-ipp-b-5451cb" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq

Started

Container started

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq

openshift-ingress

multus

payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7

AddedInterface

Add eth0 [10.133.0.91/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

AddedInterface

Add eth0 [10.133.0.94/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 6 to 10

openshift-ingress

multus

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

AddedInterface

Add eth0 [10.133.0.95/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

Unhealthy

Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

Unhealthy

Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

ProbeError

Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

ProbeError

Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

Unhealthy

Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

ProbeError

Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

ProbeError

Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

Unhealthy

Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-2864b6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-2864b6-6d848878b6 from 0 to 1
(x9)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2864b6-6d848878b6

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-2864b6-6d848878b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2864b6: serviceaccount "payload-processing-e2e-ipp-a-2864b6" not found
(x8)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2864b6: serviceaccount "payload-processing-e2e-ipp-a-2864b6" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-2864b6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-2864b6-669cd47975 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

openshift-ingress

multus

payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2864b6-6d848878b6

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.555s (2.555s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.813s (1.813s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 7 to 9

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 9; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

AddedInterface

Add eth0 [10.133.0.99/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

ProbeError

Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

Unhealthy

Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 9 to 10

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

Started

Container started

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

ProbeError

Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": read tcp 10.134.0.2:33434->10.134.0.66:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

Unhealthy

Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": read tcp 10.134.0.2:33434->10.134.0.66:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

Created

Container created
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

Unhealthy

Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

ProbeError

Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2ea125-595686979-2wc5m

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-5451cb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-5451cb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2ea125-5cc577d6b5-p45rd

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.47:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-84846d58f9-n87g7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-794c5878c5-4mnnq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-a-2ea125-openshift-default-dd6f78f5f-74lb6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-b-af8051-openshift-default-6cb5867fc5-lk6dt

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-praxis-a-918b8d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-a-918b8d

openshift-ingress

deployment-controller

e2e-praxis-a-918b8d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5

SuccessfulCreate

Created pod: e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

openshift-ingress

multus

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

AddedInterface

Add eth0 [10.133.0.111/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-5451cb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-5451cb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh
(x9)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5451cb: serviceaccount "payload-processing-e2e-ipp-b-5451cb" not found
(x8)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5451cb: serviceaccount "payload-processing-e2e-ipp-b-5451cb" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5

openshift-ingress

multus

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-af8051-79d78db899-chrvz

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5

Started

Container started
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-b-af8051-5599f954b9-g8vp8

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5

Created

Container created

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-a-918b8d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-a-918b8d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-a-918b8d-7767795f95 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-918b8d: serviceaccount "payload-processing-e2e-praxis-a-918b8d" not found

openshift-ingress

multus

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc

AddedInterface

Add eth0 [10.133.0.113/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-918b8d-7767795f95

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-918b8d-7767795f95

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-a-918b8d-7767795f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-918b8d: serviceaccount "payload-processing-e2e-praxis-a-918b8d" not found

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l

AddedInterface

Add eth0 [10.133.0.114/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-918b8d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-918b8d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

AddedInterface

Add eth0 [10.133.0.118/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-dee6aa

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-dee6aa

openshift-ingress

replicaset-controller

e2e-extmod-a-dee6aa-openshift-default-7785b59785

SuccessfulCreate

Created pod: e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

openshift-ingress

deployment-controller

e2e-extmod-a-dee6aa-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-dee6aa-openshift-default-7785b59785 from 0 to 1

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-58d898bd55 from 5 to 10

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-r5dmv

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-z24p2

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-4s7z6

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-5ljv7

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-px79s

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-5ljv7

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-r5dmv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-r5dmv

AddedInterface

Add eth0 [10.133.0.119/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-r5dmv

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-z24p2

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-4s7z6

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-px79s

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-r5dmv

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4s7z6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4s7z6

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4s7z6

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

Created

Container created
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-dee6aa: serviceaccount "payload-processing-e2e-extmod-a-dee6aa" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-dee6aa

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-dee6aa-cddd945f5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-dee6aa-cddd945f5

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-dee6aa

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-dee6aa-cddd945f5

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-dee6aa-cddd945f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-dee6aa: serviceaccount "payload-processing-e2e-extmod-a-dee6aa" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

AddedInterface

Add eth0 [10.133.0.120/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

AddedInterface

Add eth0 [10.133.0.121/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-r5dmv

ProbeError

Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

Unhealthy

Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

ProbeError

Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

Unhealthy

Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

ProbeError

Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-r5dmv

Unhealthy

Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4s7z6

Unhealthy

Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4s7z6

ProbeError

Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

ProbeError

Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

ProbeError

Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

Unhealthy

Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

Unhealthy

Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-4r82g
(x2)

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

Created

Container created

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

Unhealthy

Readiness probe failed: Get "http://10.133.0.111:15021/healthz/ready": read tcp 10.133.0.2:44380->10.133.0.111:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

ProbeError

Readiness probe error: Get "http://10.133.0.111:15021/healthz/ready": read tcp 10.133.0.2:44380->10.133.0.111:15021: read: connection reset by peer body:
(x3)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-xj4gz

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

Unhealthy

Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

ProbeError

Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-praxis-b-ca7a55

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-b-ca7a55

openshift-ingress

replicaset-controller

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5

SuccessfulCreate

Created pod: e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

openshift-ingress

deployment-controller

e2e-praxis-b-ca7a55-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5 from 0 to 1

openshift-ingress

multus

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

AddedInterface

Add eth0 [10.133.0.125/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-gwt5l

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-b-ca7a55

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9 from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

AddedInterface

Add eth0 [10.133.0.126/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-ca7a55: serviceaccount "payload-processing-e2e-praxis-b-ca7a55" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-b-ca7a55

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-ca7a55: serviceaccount "payload-processing-e2e-praxis-b-ca7a55" not found

openshift-ingress

multus

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh

Started

Container started

openshift-ingress

multus

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-918b8d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-918b8d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-ca7a55-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-ca7a55-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.312s (4.312s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dee6aa-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

ProbeError

Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

Unhealthy

Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dee6aa-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dee6aa-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-dee6aa-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-extmod-b-c6e99c-openshift-default-59687844f7

SuccessfulCreate

Created pod: e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-c6e99c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-c6e99c

openshift-ingress

multus

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-b-c6e99c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-c6e99c-openshift-default-59687844f7 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-c6e99c: serviceaccount "payload-processing-e2e-extmod-b-c6e99c" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-c6e99c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-c6e99c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-c6e99c-8b7bf895f from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-c6e99c: serviceaccount "payload-processing-e2e-extmod-b-c6e99c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

openshift-ingress

multus

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

AddedInterface

Add eth0 [10.134.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

Unhealthy

Startup probe failed: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

ProbeError

Startup probe error: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2864b6-6d848878b6-zcftx

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2864b6-669cd47975-bfwj8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-2864b6-openshift-default-864f5c8fbc-8hck6

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-ca7a55-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-ca7a55-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

Unhealthy

Readiness probe failed: Get "http://10.132.0.79:15021/healthz/ready": read tcp 10.132.0.2:51270->10.132.0.79:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

ProbeError

Readiness probe error: Get "http://10.132.0.79:15021/healthz/ready": read tcp 10.132.0.2:51270->10.132.0.79:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

Unhealthy

Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

ProbeError

Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5451cb-6cff848b7f-f7psh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5451cb-fdbf7d7f6-qwfh5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-5451cb-openshift-default-5965d59997-pgsbd

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c6e99c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c6e99c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-ck7gn

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-zbnn6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 10 to 5

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-dppk8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-6jj9z

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-2kvfj

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 2 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

Started

Container started

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

AddedInterface

Add eth0 [10.133.0.129/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

ProbeError

Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

Unhealthy

Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

Unhealthy

Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

ProbeError

Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 8; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-a48099-openshift-default-577864759 from 10 to 8

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-sgdbb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

ProbeError

Readiness probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-m5kl4

Unhealthy

Readiness probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-j7t9j

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-a48099-openshift-default-577864759 from 8 to 5

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-7gfxb

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-4rqx7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 4 to 7

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

Started

Container started

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

AddedInterface

Add eth0 [10.133.0.140/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

AddedInterface

Add eth0 [10.134.0.90/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

AddedInterface

Add eth0 [10.132.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

ProbeError

Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

ProbeError

Startup probe error: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

Unhealthy

Startup probe failed: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

Unhealthy

Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

Unhealthy

Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

ProbeError

Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-r5dmv

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-r5dmv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tn2cx

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-7g89g

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-94m8z

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-tn2cx

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-94m8z

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-7g89g

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 7 to 9

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 9; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

Started

Container started

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

AddedInterface

Add eth0 [10.133.0.144/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-gtrfd

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 5 to 4

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

Killing

Stopping container istio-proxy
(x7)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

ProbeError

Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

Unhealthy

Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

Unhealthy

Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

ProbeError

Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-4s7z6

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-4s7z6

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-z24p2

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-pgpkr

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-z24p2

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w0-afc555

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-afc555
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-918b8d-c7ffd76dc-l5npc

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

e2e-shared-a-w0-afc555-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w0-afc555-openshift-default-9b8595cd9 from 0 to 1

openshift-ingress

kubelet

e2e-praxis-a-918b8d-openshift-default-6c7bccc7d5-zs5bp

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-918b8d-7767795f95-4p84l

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9

SuccessfulCreate

Created pod: e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

openshift-ingress

multus

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

AddedInterface

Add eth0 [10.133.0.148/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w0-afc555

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-afc555: serviceaccount "payload-processing-e2e-shared-a-w0-afc555" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w0-afc555

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-afc555: serviceaccount "payload-processing-e2e-shared-a-w0-afc555" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

Started

Container started

openshift-ingress

multus

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq

AddedInterface

Add eth0 [10.134.0.92/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

AddedInterface

Add eth0 [10.134.0.91/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 9 to 10

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-ca7a55-747b56569f-5s7gh

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-ca7a55-74bd57dbd9-jrwrq

Killing

Stopping container payload-processing

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

AddedInterface

Add eth0 [10.134.0.93/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

ProbeError

Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

Unhealthy

Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-praxis-b-ca7a55-openshift-default-5fbc7f68d5-cx2k9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-ait-ec52ff90-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-ec52ff90-openshift-default-67674f5765 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-ec52ff90

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ec52ff90

openshift-ingress

replicaset-controller

e2e-ait-ec52ff90-openshift-default-67674f5765

SuccessfulCreate

Created pod: e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

openshift-ingress

multus

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

AddedInterface

Add eth0 [10.133.0.149/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-ec52ff90

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-ec52ff90-6667b4d97f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-ec52ff90

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ec52ff90-6667b4d97f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ec52ff90: serviceaccount "payload-processing-e2e-ait-ec52ff90" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ec52ff90-6667b4d97f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-ec52ff90-6667b4d97f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ec52ff90: serviceaccount "payload-processing-e2e-ait-ec52ff90" not found

openshift-ingress

multus

payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p

AddedInterface

Add eth0 [10.133.0.151/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x

AddedInterface

Add eth0 [10.133.0.150/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

ProbeError

Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

Unhealthy

Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ec52ff90-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ec52ff90-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w0-afc555-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w0-afc555-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c

SuccessfulCreate

Created pod: e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

openshift-ingress

deployment-controller

e2e-shared-b-w0-1c4191-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w0-1c4191-openshift-default-7d749f944c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w0-1c4191

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-1c4191

openshift-ingress

multus

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

AddedInterface

Add eth0 [10.134.0.95/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w0-afc555-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w0-afc555-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w0-1c4191

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w0-1c4191

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-1c4191: serviceaccount "payload-processing-e2e-shared-b-w0-1c4191" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-1c4191: serviceaccount "payload-processing-e2e-shared-b-w0-1c4191" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

AddedInterface

Add eth0 [10.133.0.152/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28

AddedInterface

Add eth0 [10.133.0.153/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

Unhealthy

Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

ProbeError

Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-p6p56

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-kf57z

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 4 to 2

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-p6p56

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4bxz8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-kf57z

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-4s6hr

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-px79s

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-8lzd5

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-58d898bd55 from 4 to 2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8lzd5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gwt5l

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-gwt5l

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

ProbeError

Readiness probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-px79s

Unhealthy

Readiness probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

Unhealthy

Readiness probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

ProbeError

Readiness probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ec52ff90-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ec52ff90-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

Unhealthy

Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

ProbeError

Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ec52ff90-7c684dbfb7-k8g5x

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ec52ff90-6667b4d97f-26q5p

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-ec52ff90-openshift-default-67674f5765-7k75j

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-migrate-c6c6fdb9-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-c6c6fdb9-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-c6c6fdb9-gw

openshift-ingress

replicaset-controller

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865

SuccessfulCreate

Created pod: e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

openshift-ingress

multus

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

AddedInterface

Add eth0 [10.133.0.157/23] from ovn-kubernetes
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w0-1c4191-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w0-1c4191-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-c6c6fdb9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-c6c6fdb9-dd8547d47 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-c6c6fdb9: serviceaccount "payload-processing-e2e-migrate-c6c6fdb9" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-c6c6fdb9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-c6c6fdb9: serviceaccount "payload-processing-e2e-migrate-c6c6fdb9" not found

openshift-ingress

kubelet

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj

Created

Container created

openshift-ingress

multus

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg

AddedInterface

Add eth0 [10.133.0.159/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj

AddedInterface

Add eth0 [10.133.0.158/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-tmwv2

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-gxmrv

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-gxmrv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-tmwv2

AddedInterface

Add eth0 [10.133.0.161/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tmwv2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tmwv2

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-gxmrv

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-c6c6fdb9-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-c6c6fdb9-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-gxmrv

AddedInterface

Add eth0 [10.133.0.160/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-gxmrv

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tmwv2

Started

Container started
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tmwv2

ProbeError

Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tmwv2

Unhealthy

Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-gxmrv

Unhealthy

Startup probe failed: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-gxmrv

ProbeError

Startup probe error: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-xsdd7

AddedInterface

Add eth0 [10.132.0.88/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-c2zj8

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Unhealthy

Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

ProbeError

Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

ProbeError

Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

Unhealthy

Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-9gnnl

AddedInterface

Add eth0 [10.132.0.90/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-kdh6t

AddedInterface

Add eth0 [10.133.0.167/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-vwd6x

AddedInterface

Add eth0 [10.134.0.99/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-zzhvc

AddedInterface

Add eth0 [10.133.0.166/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x3)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-58d898bd55 from 6 to 10

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

ProbeError

Startup probe error: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

Unhealthy

Startup probe failed: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

ProbeError

Startup probe error: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Unhealthy

Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

Unhealthy

Startup probe failed: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

ProbeError

Startup probe error: Get "http://10.133.0.166:15021/healthz/ready": dial tcp 10.133.0.166:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

Unhealthy

Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

ProbeError

Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

Unhealthy

Startup probe failed: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

ProbeError

Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-c6c6fdb9-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-c6c6fdb9-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-cf89t

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-pl9ws

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2tbkb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-wdzh4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-l62wl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-scnj7

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 10 to 4

openshift-ingress

kubelet

payload-processing-e2e-migrate-c6c6fdb9-dd8547d47-6z7wg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c6c6fdb9-796cd8c5f9-vrqjj

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

ProbeError

Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

Unhealthy

Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

Unhealthy

Readiness probe failed: Get "http://10.133.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

ProbeError

Readiness probe error: Get "http://10.133.0.167:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

Unhealthy

Readiness probe failed: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

ProbeError

Readiness probe error: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

Unhealthy

Readiness probe failed: Get "http://10.133.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

ProbeError

Readiness probe error: Get "http://10.133.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x3)

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled down replica set e2e-worker-w4-a48099-openshift-default-577864759 from 2 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulDelete

Deleted pod: e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

(combined from similar events): New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-wl8rv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc

SuccessfulDelete

Deleted pod: e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

openshift-ingress

deployment-controller

e2e-worker-w2-3d70ff-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled down replica set e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-3d70ff-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-migrate-c6c6fdb9-gw-openshift-default-78784b5865-wt8rp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-cxjqm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-ait-4cbcb62b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-4cbcb62b-openshift-default-7657bb7b8f from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f

SuccessfulCreate

Created pod: e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

openshift-ingress

gateway_labeler_controller

e2e-ait-4cbcb62b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-4cbcb62b

openshift-ingress

multus

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg

openshift-ingress

deployment-controller

payload-processing-e2e-ait-4cbcb62b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-4cbcb62b-849d7dcbc9 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9

FailedCreate

Error creating: pods "payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4cbcb62b: serviceaccount "payload-processing-e2e-ait-4cbcb62b" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-4cbcb62b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4cbcb62b: serviceaccount "payload-processing-e2e-ait-4cbcb62b" not found

openshift-ingress

kubelet

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls

AddedInterface

Add eth0 [10.134.0.102/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg

AddedInterface

Add eth0 [10.134.0.101/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls

Created

Container created

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9

SuccessfulCreate

Created pod: payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls

openshift-ingress

replicaset-controller

e2e-mt-a08eb536-openshift-default-6775d7bbcc

SuccessfulCreate

Created pod: e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

openshift-ingress

deployment-controller

e2e-mt-a08eb536-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-a08eb536-openshift-default-6775d7bbcc from 0 to 1

openshift-ingress

multus

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

AddedInterface

Add eth0 [10.132.0.91/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-mt-a08eb536

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a08eb536

openshift-ingress

deployment-controller

payload-processing-e2e-mt-a08eb536

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-a08eb536-768db898c5 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-a08eb536-768db898c5

FailedCreate

Error creating: pods "payload-processing-e2e-mt-a08eb536-768db898c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a08eb536: serviceaccount "payload-processing-e2e-mt-a08eb536" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-a08eb536-768db898c5

SuccessfulCreate

Created pod: payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-a08eb536

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-a08eb536-6fc8fc647 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a08eb536: serviceaccount "payload-processing-e2e-mt-a08eb536" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb

AddedInterface

Add eth0 [10.133.0.170/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2

AddedInterface

Add eth0 [10.134.0.103/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2

Started

Container started

opendatahub

kubelet

test-curl-7440-fed722

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

Created

Container created

opendatahub

kubelet

test-curl-7440-fed722

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7440-fed722

AddedInterface

Add eth0 [10.132.0.93/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7440-fed722

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7440-fed722

Killing

Stopping container test-curl-7440-fed722
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4cbcb62b-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4cbcb62b-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

Unhealthy

Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

ProbeError

Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4cbcb62b-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4cbcb62b-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4cbcb62b-5d57bdcc59-vpvfg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-4cbcb62b-849d7dcbc9-vjtls

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-a08eb536-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-a08eb536-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-api-a-faa8ff

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-faa8ff

openshift-ingress

multus

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

AddedInterface

Add eth0 [10.133.0.176/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-a-faa8ff-openshift-default-667f695f57

SuccessfulCreate

Created pod: e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

openshift-ingress

deployment-controller

e2e-api-a-faa8ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-faa8ff-openshift-default-667f695f57 from 0 to 1
(x6)

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

ProbeError

Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

Unhealthy

Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-faa8ff

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-faa8ff: serviceaccount "payload-processing-e2e-api-a-faa8ff" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-faa8ff-d4bb67df9

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-faa8ff-d4bb67df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-faa8ff: serviceaccount "payload-processing-e2e-api-a-faa8ff" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-faa8ff

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-faa8ff-d4bb67df9 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf

openshift-ingress

multus

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf

AddedInterface

Add eth0 [10.133.0.178/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq

Created

Container created

openshift-ingress

multus

payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq

AddedInterface

Add eth0 [10.133.0.177/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-faa8ff-d4bb67df9

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq

openshift-ingress

kubelet

payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf

Started

Container started

openshift-ingress

kubelet

e2e-ait-4cbcb62b-openshift-default-7657bb7b8f-tt664

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ait-ad90c52d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-ad90c52d-openshift-default-6bc76dbd85 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-ad90c52d-openshift-default-6bc76dbd85

SuccessfulCreate

Created pod: e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

openshift-ingress

gateway_labeler_controller

e2e-ait-ad90c52d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ad90c52d
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-a08eb536-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

AddedInterface

Add eth0 [10.133.0.180/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-a08eb536-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

payload-processing-e2e-ait-ad90c52d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-ad90c52d-d779bc747 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-ad90c52d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-ad90c52d-7b64b785f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ad90c52d-d779bc747

SuccessfulCreate

Created pod: payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ad90c52d-d779bc747

FailedCreate

Error creating: pods "payload-processing-e2e-ait-ad90c52d-d779bc747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ad90c52d: serviceaccount "payload-processing-e2e-ait-ad90c52d" not found
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ad90c52d: serviceaccount "payload-processing-e2e-ait-ad90c52d" not found

openshift-ingress

multus

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5

AddedInterface

Add eth0 [10.134.0.107/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5

Created

Container created

openshift-ingress

multus

payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg

AddedInterface

Add eth0 [10.134.0.106/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled down replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 4 to 3

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-ld5mx

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 3 to 2

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-n8g9t

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

ProbeError

Startup probe error: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

Unhealthy

Startup probe failed: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-faa8ff-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-faa8ff-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

ProbeError

Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

Unhealthy

Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ad90c52d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ad90c52d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-mt-a08eb536-768db898c5-4m5l2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-a08eb536-6fc8fc647-2pmhb

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-faa8ff-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-faa8ff-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ad90c52d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-ad90c52d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-api-b-8815d2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-8815d2-openshift-default-68fcb7d459 from 0 to 1

openshift-ingress

multus

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

AddedInterface

Add eth0 [10.134.0.112/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-b-8815d2-openshift-default-68fcb7d459

SuccessfulCreate

Created pod: e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

openshift-ingress

gateway_labeler_controller

e2e-api-b-8815d2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-8815d2

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-8815d2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-8815d2-6b474787f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-8815d2-6b474787f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-8815d2-6b474787f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-8815d2-6b474787f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-8815d2: serviceaccount "payload-processing-e2e-api-b-8815d2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-8815d2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-8815d2-58457b5f85 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-8815d2-58457b5f85

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-8815d2-58457b5f85

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-8815d2-58457b5f85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-8815d2: serviceaccount "payload-processing-e2e-api-b-8815d2" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh

AddedInterface

Add eth0 [10.134.0.113/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf

AddedInterface

Add eth0 [10.133.0.190/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf

Created

Container created

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-a08eb536-openshift-default-6775d7bbcc-sjh5g

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tmwv2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zzhvc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-gxmrv

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-tmwv2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kdh6t

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-mt-25565930

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-25565930

openshift-ingress

kubelet

payload-processing-e2e-ait-ad90c52d-d779bc747-ngwgg

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-mt-25565930-openshift-default-64f5666649

SuccessfulCreate

Created pod: e2e-mt-25565930-openshift-default-64f5666649-rq68b

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ad90c52d-7b64b785f-kn8k5

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-mt-25565930-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-25565930-openshift-default-64f5666649 from 0 to 1

openshift-ingress

multus

e2e-mt-25565930-openshift-default-64f5666649-rq68b

AddedInterface

Add eth0 [10.133.0.192/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-mt-25565930

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-25565930-5c94467bfc from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-25565930-5c94467bfc

FailedCreate

Error creating: pods "payload-processing-e2e-mt-25565930-5c94467bfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-25565930: serviceaccount "payload-processing-e2e-mt-25565930" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-25565930-b8577d659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-25565930-b8577d659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-25565930: serviceaccount "payload-processing-e2e-mt-25565930" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-25565930

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-25565930-b8577d659 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-25565930-5c94467bfc

SuccessfulCreate

Created pod: payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-25565930-b8577d659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

openshift-ingress

multus

payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

AddedInterface

Add eth0 [10.133.0.193/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

Started

Container started

openshift-ingress

multus

payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

AddedInterface

Add eth0 [10.133.0.194/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-ad90c52d-openshift-default-6bc76dbd85-htpzt

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-derive-31e728ff

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-31e728ff

openshift-ingress

deployment-controller

e2e-derive-31e728ff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-31e728ff-openshift-default-6f8f67d765 from 0 to 1

openshift-ingress

replicaset-controller

e2e-derive-31e728ff-openshift-default-6f8f67d765

SuccessfulCreate

Created pod: e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

openshift-ingress

multus

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

AddedInterface

Add eth0 [10.132.0.96/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-31e728ff: serviceaccount "payload-processing-e2e-derive-31e728ff" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-31e728ff-b8958465f

FailedCreate

Error creating: pods "payload-processing-e2e-derive-31e728ff-b8958465f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-31e728ff: serviceaccount "payload-processing-e2e-derive-31e728ff" not found

openshift-ingress

deployment-controller

payload-processing-e2e-derive-31e728ff

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-31e728ff-b8958465f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-31e728ff

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f

AddedInterface

Add eth0 [10.133.0.199/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr

AddedInterface

Add eth0 [10.133.0.200/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f

Created

Container created

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-31e728ff-b8958465f

SuccessfulCreate

Created pod: payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr

openshift-ingress

kubelet

payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

Unhealthy

Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

ProbeError

Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-8815d2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-8815d2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-8815d2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-8815d2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-rq68b

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-rq68b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-rq68b

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-rq68b

ProbeError

Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-rq68b

Unhealthy

Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-67d67f47f4 from 0 to 1
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-67d67f47f4

FailedCreate

Error creating: pods "payload-processing-external-model-67d67f47f4-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

openshift-ingress

kubelet

payload-processing-e2e-api-a-faa8ff-d4bb67df9-zd2sq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-faa8ff-5b96b764b7-l9wrf

Killing

Stopping container payload-pre-processing

models-as-a-service

multus

payload-processing-external-model-67d67f47f4-bp4gq

AddedInterface

Add eth0 [10.133.0.215/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-67d67f47f4

SuccessfulCreate

Created pod: payload-processing-external-model-67d67f47f4-bp4gq

models-as-a-service

kubelet

payload-processing-external-model-67d67f47f4-bp4gq

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-67d67f47f4-bp4gq

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-67d67f47f4-bp4gq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-25565930-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-25565930-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-31e728ff-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-25565930-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-25565930-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-31e728ff-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

Unhealthy

Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

ProbeError

Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-ztp4p

AddedInterface

Add eth0 [10.133.0.220/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-58d898bd55-mpdzz

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-mpdzz

AddedInterface

Add eth0 [10.133.0.218/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-zml98

AddedInterface

Add eth0 [10.134.0.116/23] from ovn-kubernetes

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-58d898bd55 from 6 to 10

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-sd9zb

AddedInterface

Add eth0 [10.133.0.219/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-sd9zb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-sd9zb

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-sd9zb

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

ProbeError

Startup probe error: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-sd9zb

Unhealthy

Startup probe failed: Get "http://10.133.0.219:15021/healthz/ready": dial tcp 10.133.0.219:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-sd9zb

ProbeError

Startup probe error: Get "http://10.133.0.219:15021/healthz/ready": dial tcp 10.133.0.219:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

ProbeError

Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-api-a-faa8ff-openshift-default-667f695f57-982wc

Killing

Stopping container istio-proxy
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

ProbeError

Startup probe error: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

Unhealthy

Startup probe failed: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

Unhealthy

Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

Unhealthy

Startup probe failed: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-8815d2-6b474787f-dcrwf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-8815d2-58457b5f85-t72rh

Killing

Stopping container payload-processing
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-31e728ff-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-31e728ff-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

models-as-a-service

kubelet

payload-processing-external-model-67d67f47f4-bp4gq

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-59599f94ff from 0 to 1
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-59599f94ff

FailedCreate

Error creating: pods "payload-processing-external-model-59599f94ff-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-59599f94ff

SuccessfulCreate

Created pod: payload-processing-external-model-59599f94ff-6qzvs

models-as-a-service

multus

payload-processing-external-model-59599f94ff-6qzvs

AddedInterface

Add eth0 [10.133.0.223/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-59599f94ff-6qzvs

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-59599f94ff-6qzvs

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-59599f94ff-6qzvs

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-59599f94ff-6qzvs

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-31e728ff-b8958465f-cxf5f

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-31e728ff-76fbbf7c79-xwwmr

Killing

Stopping container payload-pre-processing
(x4)

models-as-a-service

kubelet

payload-processing-external-model-59599f94ff-6qzvs

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-d27da752f422" not found
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-59599f94ff

FailedCreate

Error creating: pods "payload-processing-external-model-59599f94ff-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-59599f94ff from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-59599f94ff

SuccessfulCreate

Created pod: payload-processing-external-model-59599f94ff-dzx6z
(x3)

models-as-a-service

kubelet

payload-processing-external-model-59599f94ff-dzx6z

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-d27da752f422" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5565547b4b from 0 to 1
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-5565547b4b

FailedCreate

Error creating: pods "payload-processing-external-model-5565547b4b-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

kubelet

payload-processing-external-model-5565547b4b-2kj8t

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-5565547b4b-2kj8t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-5565547b4b

SuccessfulCreate

Created pod: payload-processing-external-model-5565547b4b-2kj8t

models-as-a-service

kubelet

payload-processing-external-model-5565547b4b-2kj8t

Created

Container created

models-as-a-service

multus

payload-processing-external-model-5565547b4b-2kj8t

AddedInterface

Add eth0 [10.133.0.225/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5565547b4b-2kj8t

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-autores-ad05c6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-ad05c6

openshift-ingress

multus

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

AddedInterface

Add eth0 [10.133.0.226/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-autores-ad05c6-openshift-default-5fdb7744c7

SuccessfulCreate

Created pod: e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

openshift-ingress

deployment-controller

e2e-autores-ad05c6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-ad05c6-openshift-default-5fdb7744c7 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-ad05c6: serviceaccount "payload-processing-e2e-autores-ad05c6" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-ad05c6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-ad05c6-697dd9486c

FailedCreate

Error creating: pods "payload-processing-e2e-autores-ad05c6-697dd9486c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-ad05c6: serviceaccount "payload-processing-e2e-autores-ad05c6" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-ad05c6-697dd9486c

SuccessfulCreate

Created pod: payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc

openshift-ingress

deployment-controller

payload-processing-e2e-autores-ad05c6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-ad05c6-697dd9486c from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc

AddedInterface

Add eth0 [10.133.0.227/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc

Started

Container started

openshift-ingress

multus

payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc

AddedInterface

Add eth0 [10.133.0.228/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc

Created

Container created

openshift-ingress

kubelet

e2e-api-b-8815d2-openshift-default-68fcb7d459-wqcrd

Killing

Stopping container istio-proxy

opendatahub

kubelet

test-curl-7440-705aa1

Started

Container started

opendatahub

kubelet

test-curl-7440-705aa1

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7440-705aa1

AddedInterface

Add eth0 [10.132.0.102/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7440-705aa1

Created

Container created

opendatahub

kubelet

test-curl-7440-705aa1

Killing

Stopping container test-curl-7440-705aa1

opendatahub

kubelet

test-curl-7440-3ab145

Started

Container started

opendatahub

multus

test-curl-7440-3ab145

AddedInterface

Add eth0 [10.132.0.103/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7440-3ab145

Created

Container created

opendatahub

kubelet

test-curl-7440-3ab145

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7440-3ab145

Killing

Stopping container test-curl-7440-3ab145

opendatahub

multus

test-curl-7440-108a1d

AddedInterface

Add eth0 [10.132.0.104/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7440-108a1d

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7440-108a1d

Created

Container created

opendatahub

kubelet

test-curl-7440-108a1d

Started

Container started

opendatahub

kubelet

test-curl-7440-108a1d

Killing

Stopping container test-curl-7440-108a1d

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

Killing

Stopping container istio-proxy

opendatahub

kubelet

test-curl-7440-1d3127

Created

Container created

opendatahub

kubelet

test-curl-7440-1d3127

Started

Container started

opendatahub

kubelet

test-curl-7440-1d3127

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7440-1d3127

AddedInterface

Add eth0 [10.132.0.105/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7440-1d3127

Killing

Stopping container test-curl-7440-1d3127

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-derive-31e728ff-openshift-default-6f8f67d765-dd698

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

e2e-extmod-a-c73135-openshift-default-69c4df7c97

SuccessfulCreate

Created pod: e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

openshift-ingress

deployment-controller

e2e-extmod-a-c73135-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-c73135-openshift-default-69c4df7c97 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-c73135

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-c73135

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-57896c8f89 from 0 to 1

openshift-ingress

multus

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

AddedInterface

Add eth0 [10.134.0.117/23] from ovn-kubernetes
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-57896c8f89

FailedCreate

Error creating: pods "payload-processing-external-model-57896c8f89-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

multus

payload-processing-external-model-57896c8f89-hx4zg

AddedInterface

Add eth0 [10.134.0.118/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-57896c8f89

SuccessfulCreate

Created pod: payload-processing-external-model-57896c8f89-hx4zg

models-as-a-service

kubelet

payload-processing-external-model-57896c8f89-hx4zg

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-57896c8f89-hx4zg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-57896c8f89-hx4zg

Started

Container started

opendatahub

kubelet

test-curl-7446-a58b91

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7446-a58b91

Started

Container started

opendatahub

kubelet

test-curl-7446-a58b91

Created

Container created

opendatahub

multus

test-curl-7446-a58b91

AddedInterface

Add eth0 [10.132.0.106/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-c73135

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-c73135-7575bf697f from 0 to 1

opendatahub

kubelet

test-curl-7446-a58b91

Killing

Stopping container test-curl-7446-a58b91

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-c73135

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-c73135: serviceaccount "payload-processing-e2e-extmod-a-c73135" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-c73135-7575bf697f

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-c73135-7575bf697f

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-c73135-7575bf697f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-c73135: serviceaccount "payload-processing-e2e-extmod-a-c73135" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx

AddedInterface

Add eth0 [10.133.0.230/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn

AddedInterface

Add eth0 [10.133.0.229/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c73135-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c73135-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-848b684699 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-848b684699

SuccessfulCreate

Created pod: payload-processing-external-model-848b684699-zqxjt

models-as-a-service

kubelet

payload-processing-external-model-848b684699-zqxjt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-848b684699-zqxjt

AddedInterface

Add eth0 [10.134.0.119/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-848b684699-zqxjt

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-848b684699-zqxjt

Started

Container started
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-gqg7j

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-lzl6m

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

ProbeError

Readiness probe error: Get "http://10.133.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

Unhealthy

Readiness probe failed: Get "http://10.133.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

Unhealthy

Startup probe failed: Get "http://10.133.0.226:15021/healthz/ready": dial tcp 10.133.0.226:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

ProbeError

Startup probe error: Get "http://10.133.0.226:15021/healthz/ready": dial tcp 10.133.0.226:15021: connect: connection refused body:

models-as-a-service

kubelet

payload-processing-external-model-848b684699-zqxjt

Killing

Stopping container payload-processing
(x4)

models-as-a-service

kubelet

payload-processing-external-model-848b684699-zqxjt

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-848b684699 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-ff8dd698

SuccessfulCreate

Created pod: payload-processing-external-model-ff8dd698-cvrtg

models-as-a-service

replicaset-controller

payload-processing-external-model-848b684699

SuccessfulDelete

Deleted pod: payload-processing-external-model-848b684699-zqxjt

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-cvrtg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-ff8dd698-cvrtg

AddedInterface

Add eth0 [10.133.0.235/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-cvrtg

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-cvrtg

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

Started

Container started
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-ad05c6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-ad05c6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

Unhealthy

Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

ProbeError

Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused body:

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-cvrtg

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6c9f596676 from 0 to 1

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-ff8dd698 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-ff8dd698

SuccessfulDelete

Deleted pod: payload-processing-external-model-ff8dd698-cvrtg

models-as-a-service

replicaset-controller

payload-processing-external-model-6c9f596676

SuccessfulCreate

Created pod: payload-processing-external-model-6c9f596676-2bds7
(x3)

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-cvrtg

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-6c9f596676-2bds7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-6c9f596676-2bds7

Started

Container started

models-as-a-service

multus

payload-processing-external-model-6c9f596676-2bds7

AddedInterface

Add eth0 [10.133.0.236/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6c9f596676-2bds7

Created

Container created
(x6)

models-as-a-service

kubelet

payload-processing-external-model-57896c8f89-hx4zg

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-0c80b87999df" not found

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

Unhealthy

Readiness probe failed: Get "http://10.134.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

ProbeError

Readiness probe error: Get "http://10.134.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

models-as-a-service

replicaset-controller

payload-processing-external-model-57896c8f89

SuccessfulDelete

Deleted pod: payload-processing-external-model-57896c8f89-hx4zg

models-as-a-service

kubelet

payload-processing-external-model-57896c8f89-hx4zg

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-57896c8f89 from 1 to 0
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c73135-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c73135-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-7a360a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-7a360a

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulCreate

Created pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

openshift-ingress

deployment-controller

e2e-extmod-b-7a360a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-7a360a-openshift-default-5dd9fb679c from 0 to 1

openshift-ingress

multus

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

AddedInterface

Add eth0 [10.133.0.237/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-7a360a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-7a360a: serviceaccount "payload-processing-e2e-extmod-b-7a360a" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-7a360a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-7a360a-6cf998897 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-7a360a-6cf998897

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-7a360a-6cf998897

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-7a360a-6cf998897-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-7a360a: serviceaccount "payload-processing-e2e-extmod-b-7a360a" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9

AddedInterface

Add eth0 [10.134.0.120/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7

AddedInterface

Add eth0 [10.134.0.121/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-mt-25565930

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-25565930-5c94467bfc from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj

AddedInterface

Add eth0 [10.134.0.122/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-25565930-b8577d659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-25565930

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-25565930-b8577d659 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-25565930-b8577d659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-25565930-b8577d659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-25565930: serviceaccount "payload-processing-e2e-mt-25565930" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-25565930-5c94467bfc

SuccessfulCreate

Created pod: payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-25565930-5c94467bfc

FailedCreate

Error creating: pods "payload-processing-e2e-mt-25565930-5c94467bfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-25565930: serviceaccount "payload-processing-e2e-mt-25565930" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt

Created

Container created

openshift-ingress

multus

payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt

AddedInterface

Add eth0 [10.134.0.123/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-processing-e2e-autores-ad05c6-697dd9486c-56jfc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ad05c6-6b9b5d9fff-zswjc

Killing

Stopping container payload-pre-processing

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-ad05c6-openshift-default

FailedUpdateStatus

horizontalpodautoscalers.autoscaling "e2e-autores-ad05c6-openshift-default" not found

openshift-ingress

kubelet

e2e-autores-ad05c6-openshift-default-5fdb7744c7-469rp

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

AddedInterface

Add eth0 [10.132.0.112/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-explicit-5d585e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-5d585e-openshift-default-848cc669d9 from 0 to 1

openshift-ingress

replicaset-controller

e2e-explicit-5d585e-openshift-default-848cc669d9

SuccessfulCreate

Created pod: e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

openshift-ingress

gateway_labeler_controller

e2e-explicit-5d585e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-5d585e

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-25565930-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-mt-25565930-openshift-default-64f5666649

SuccessfulCreate

Created pod: e2e-mt-25565930-openshift-default-64f5666649-wjfgh

openshift-ingress

replicaset-controller

e2e-mt-25565930-openshift-default-64f5666649

SuccessfulCreate

Created pod: e2e-mt-25565930-openshift-default-64f5666649-gmkbz

openshift-ingress

replicaset-controller

e2e-mt-25565930-openshift-default-64f5666649

SuccessfulCreate

Created pod: e2e-mt-25565930-openshift-default-64f5666649-gbhn9

openshift-ingress

deployment-controller

e2e-mt-25565930-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-25565930-openshift-default-64f5666649 from 1 to 4

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

Started

Container started

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

Started

Container started

openshift-ingress

multus

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

AddedInterface

Add eth0 [10.133.0.238/23] from ovn-kubernetes

openshift-ingress

multus

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

AddedInterface

Add eth0 [10.134.0.124/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

Started

Container started

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

Created

Container created

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

Created

Container created

openshift-ingress

multus

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

AddedInterface

Add eth0 [10.132.0.113/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

Created

Container created
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-5d585e-687cb4555c

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-5d585e-687cb4555c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-5d585e: serviceaccount "payload-processing-e2e-explicit-5d585e" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-5d585e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-5d585e-5746b6dc67 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-5d585e: serviceaccount "payload-processing-e2e-explicit-5d585e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-5d585e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-5d585e-687cb4555c from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulCreate

Created pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulCreate

Created pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

openshift-ingress

multus

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

AddedInterface

Add eth0 [10.132.0.114/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

Started

Container started

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulCreate

Created pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl

AddedInterface

Add eth0 [10.133.0.240/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc

AddedInterface

Add eth0 [10.133.0.239/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-5d585e-687cb4555c

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl

openshift-ingress

deployment-controller

e2e-extmod-b-7a360a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-7a360a-openshift-default-5dd9fb679c from 1 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

AddedInterface

Add eth0 [10.134.0.125/23] from ovn-kubernetes

openshift-ingress

multus

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

AddedInterface

Add eth0 [10.133.0.241/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

ProbeError

Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

Unhealthy

Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

ProbeError

Startup probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

Unhealthy

Startup probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

Unhealthy

Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

ProbeError

Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

ProbeError

Startup probe error: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

ProbeError

Startup probe error: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

Unhealthy

Startup probe failed: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

ProbeError

Startup probe error: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

Unhealthy

Startup probe failed: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

Unhealthy

Startup probe failed: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

ProbeError

Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

Unhealthy

Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-mt-25565930-openshift-default-64f5666649

SuccessfulCreate

Created pod: e2e-mt-25565930-openshift-default-64f5666649-5crwq

openshift-ingress

replicaset-controller

e2e-mt-25565930-openshift-default-64f5666649

SuccessfulCreate

Created pod: e2e-mt-25565930-openshift-default-64f5666649-v8m4h

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-25565930-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-mt-25565930-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-25565930-openshift-default-64f5666649 from 4 to 7

openshift-ingress

replicaset-controller

e2e-mt-25565930-openshift-default-64f5666649

SuccessfulCreate

Created pod: e2e-mt-25565930-openshift-default-64f5666649-cpzlj

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-5crwq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

AddedInterface

Add eth0 [10.134.0.126/23] from ovn-kubernetes

openshift-ingress

multus

e2e-mt-25565930-openshift-default-64f5666649-5crwq

AddedInterface

Add eth0 [10.133.0.242/23] from ovn-kubernetes

openshift-ingress

multus

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

AddedInterface

Add eth0 [10.132.0.115/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

Created

Container created

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

Started

Container started

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-5crwq

Created

Container created

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-5crwq

Started

Container started

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

Created

Container created

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

Started

Container started

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

ProbeError

Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-5crwq

Unhealthy

Startup probe failed: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-5crwq

ProbeError

Startup probe error: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

ProbeError

Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

Unhealthy

Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

Unhealthy

Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulCreate

Created pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulCreate

Created pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-extmod-b-7a360a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-7a360a-openshift-default-5dd9fb679c from 4 to 6

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

AddedInterface

Add eth0 [10.132.0.116/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

Started

Container started

openshift-ingress

multus

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

AddedInterface

Add eth0 [10.134.0.127/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 2 to 1

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulDelete

Deleted pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-2j4vw

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

Unhealthy

Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

Unhealthy

Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

ProbeError

Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

ProbeError

Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused body:

opendatahub

multus

test-curl-7446-9f4e17

AddedInterface

Add eth0 [10.133.0.244/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7446-9f4e17

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

kubelet

test-curl-7446-9f4e17

Created

Container created

opendatahub

kubelet

test-curl-7446-9f4e17

Started

Container started

opendatahub

kubelet

test-curl-7446-9f4e17

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.941s (2.941s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7446-9f4e17

Killing

Stopping container test-curl-7446-9f4e17
(x2)

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

ProbeError

Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

Unhealthy

Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-25565930-b8577d659-6pwlj

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-25565930-5c94467bfc-hp4rt

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-dfp4l

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-dfp4l

Created

Container created
(x2)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-ff8dd698 from 0 to 1

models-as-a-service

multus

payload-processing-external-model-ff8dd698-dfp4l

AddedInterface

Add eth0 [10.133.0.247/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-dfp4l

Started

Container started

models-as-a-service

replicaset-controller

payload-processing-external-model-ff8dd698

SuccessfulCreate

Created pod: payload-processing-external-model-ff8dd698-dfp4l
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-5d585e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-5d585e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-v8m4h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-wjfgh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-cpzlj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-rq68b

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-5crwq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gmkbz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-25565930-openshift-default-64f5666649-gbhn9

Killing

Stopping container istio-proxy

models-as-a-service

kubelet

payload-processing-external-model-6c9f596676-2bds7

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-dfp4l

Killing

Stopping container payload-processing
(x2)

models-as-a-service

kubelet

payload-processing-external-model-ff8dd698-dfp4l

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-d96a66

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-d96a66

openshift-ingress

replicaset-controller

e2e-extmod-a-d96a66-openshift-default-545c685db5

SuccessfulCreate

Created pod: e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

openshift-ingress

deployment-controller

e2e-extmod-a-d96a66-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-d96a66-openshift-default-545c685db5 from 0 to 1
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-5d585e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

AddedInterface

Add eth0 [10.133.0.250/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-5d585e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-d96a66

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-d96a66: serviceaccount "payload-processing-e2e-extmod-a-d96a66" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-d96a66

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-d96a66-6457d7f547 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-d96a66-6457d7f547

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-d96a66-6457d7f547

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-d96a66-6457d7f547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-d96a66: serviceaccount "payload-processing-e2e-extmod-a-d96a66" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

AddedInterface

Add eth0 [10.133.0.253/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd

AddedInterface

Add eth0 [10.133.0.252/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Unhealthy

Readiness probe failed: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

ProbeError

Readiness probe error: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Unhealthy

Readiness probe failed: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

ProbeError

Readiness probe error: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5d585e-5746b6dc67-ql7vc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-5d585e-687cb4555c-dgwpl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-noten-40af1b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-40af1b

openshift-ingress

replicaset-controller

e2e-noten-40af1b-openshift-default-79966c65b5

SuccessfulCreate

Created pod: e2e-noten-40af1b-openshift-default-79966c65b5-lt94f

openshift-ingress

deployment-controller

e2e-noten-40af1b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-40af1b-openshift-default-79966c65b5 from 0 to 1

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-noten-40af1b-openshift-default-79966c65b5-lt94f

Started

Container started

openshift-ingress

kubelet

e2e-noten-40af1b-openshift-default-79966c65b5-lt94f

Created

Container created

openshift-ingress

multus

e2e-noten-40af1b-openshift-default-79966c65b5-lt94f

AddedInterface

Add eth0 [10.133.1.3/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-explicit-5d585e-openshift-default-848cc669d9-ncktr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-noten-40af1b-openshift-default-79966c65b5-lt94f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-098e4d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-098e4d

openshift-ingress

deployment-controller

e2e-extmod-b-098e4d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-098e4d-openshift-default-dbccdd77c from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-098e4d-openshift-default-dbccdd77c

SuccessfulCreate

Created pod: e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

openshift-ingress

multus

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

AddedInterface

Add eth0 [10.133.1.4/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-d96a66-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-d96a66-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-40af1b-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-40af1b-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-098e4d-58d9df76b9

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-098e4d-58d9df76b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-098e4d: serviceaccount "payload-processing-e2e-extmod-b-098e4d" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-098e4d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-098e4d: serviceaccount "payload-processing-e2e-extmod-b-098e4d" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-098e4d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-098e4d-58d9df76b9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-098e4d-58d9df76b9

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

AddedInterface

Add eth0 [10.133.1.5/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh

AddedInterface

Add eth0 [10.133.1.6/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

ProbeError

Startup probe error: Get "http://10.133.0.250:15021/healthz/ready": dial tcp 10.133.0.250:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

Unhealthy

Startup probe failed: Get "http://10.133.0.250:15021/healthz/ready": dial tcp 10.133.0.250:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

Created

Container created

openshift-ingress

kubelet

e2e-noten-40af1b-openshift-default-79966c65b5-lt94f

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-extmod-b-7a360a-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-7a360a-openshift-default-5dd9fb679c from 6 to 5

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulDelete

Deleted pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-ttzqr

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-d96a66-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-d96a66-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f

SuccessfulCreate

Created pod: e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

openshift-ingress

deployment-controller

e2e-shared-a-w1-897c87-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w1-897c87

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-897c87

openshift-ingress

multus

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

AddedInterface

Add eth0 [10.132.0.119/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-897c87-57666cd899

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w1-897c87-57666cd899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-897c87: serviceaccount "payload-processing-e2e-shared-a-w1-897c87" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w1-897c87

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w1-897c87-57666cd899 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-897c87-57666cd899

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w1-897c87

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-897c87: serviceaccount "payload-processing-e2e-shared-a-w1-897c87" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5

Created

Container created

openshift-ingress

multus

payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

AddedInterface

Add eth0 [10.133.1.8/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5

AddedInterface

Add eth0 [10.133.1.7/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulDelete

Deleted pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

openshift-ingress

deployment-controller

e2e-extmod-b-7a360a-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-7a360a-openshift-default-5dd9fb679c from 5 to 4

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-4mgxt

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

ProbeError

Startup probe error: Get "http://10.133.1.4:15021/healthz/ready": dial tcp 10.133.1.4:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

Unhealthy

Startup probe failed: Get "http://10.133.1.4:15021/healthz/ready": dial tcp 10.133.1.4:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-098e4d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-098e4d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-extmod-b-7a360a-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-7a360a-openshift-default-5dd9fb679c from 4 to 2

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulDelete

Deleted pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulDelete

Deleted pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jnbcc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-jm8wx

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-098e4d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-098e4d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w1-897c87-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w1-897c87-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

Unhealthy

Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

ProbeError

Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w1-897c87-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w1-897c87-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-shared-b-w1-fd2672-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w1-fd2672

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-fd2672

openshift-ingress

replicaset-controller

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69

SuccessfulCreate

Created pod: e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

openshift-ingress

multus

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

AddedInterface

Add eth0 [10.132.0.121/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w1-fd2672

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w1-fd2672-659f849fc9 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w1-fd2672

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-fd2672: serviceaccount "payload-processing-e2e-shared-b-w1-fd2672" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-fd2672: serviceaccount "payload-processing-e2e-shared-b-w1-fd2672" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

AddedInterface

Add eth0 [10.134.0.132/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

openshift-ingress

multus

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

AddedInterface

Add eth0 [10.134.0.133/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-3d70ff-6dc999899-r9nvw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-3d70ff-5df597f9cf-7v76d

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w2-3d70ff-openshift-default-7c7dc9f4fc-sgrjg

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-fchpl
(x2)

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 1 to 2

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

Started

Container started

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

AddedInterface

Add eth0 [10.134.0.137/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

Unhealthy

Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

ProbeError

Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

Unhealthy

Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

ProbeError

Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w1-fd2672-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w1-fd2672-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-58d898bd55 from 10 to 6

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-c2zj8

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-vwd6x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c2zj8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-5ljv7

Killing

Stopping container istio-proxy
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-vwd6x

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-5ljv7

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-zml98

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-7a360a-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-extmod-b-7a360a-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-7a360a-openshift-default-5dd9fb679c from 2 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c

SuccessfulDelete

Deleted pod: e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-wqklt

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx

Killing

Stopping container payload-processing
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w1-fd2672-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-qpjb7
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w1-fd2672-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 2 to 3

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

Created

Container created

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

AddedInterface

Add eth0 [10.132.0.126/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

Unhealthy

Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

ProbeError

Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

ProbeError

Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

Unhealthy

Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c73135-78b44ccf5-884wn

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c73135-7575bf697f-mstqx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-a-c73135-openshift-default-69c4df7c97-mhljx

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-dee6aa-cddd945f5-lmc2z

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-dee6aa-864f9cfc9f-ljqpd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-trkbg

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-5bms2
(x2)

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 3 to 5
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-7a360a-6cf998897-7v6d7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

AddedInterface

Add eth0 [10.134.0.139/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

AddedInterface

Add eth0 [10.133.1.36/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-7a360a-d4b7c96cf-cswr9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

Created

Container created
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-d96a66-6457d7f547-65qrf

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-d96a66-64c48bbc79-shfkd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

ProbeError

Startup probe error: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

Unhealthy

Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

ProbeError

Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

Unhealthy

Startup probe failed: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-d96a66-openshift-default-545c685db5-c5gdn

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-dee6aa-openshift-default-7785b59785-gt2w7

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-098e4d-5f8b9f6575-lngfh

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-098e4d-58d9df76b9-7drbt

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-b-c6e99c-openshift-default-59687844f7-tzqft

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-7a360a-openshift-default-5dd9fb679c-xww2p

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c6e99c-8b7bf895f-hj225

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c6e99c-fcb666fb7-kc9cw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-b-098e4d-openshift-default-dbccdd77c-ptlf2

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-afc555-84db8c5b8c-kh4bq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-afc555-86567dff5-4gjlw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-a48099-6566d45677-7kcpx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-a48099-786fbc588c-zpqsp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-a-w0-afc555-openshift-default-9b8595cd9-zwkgk

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-worker-w4-a48099-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-a48099-openshift-default-577864759 from 5 to 10
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-a48099-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-nkc44

openshift-ingress

replicaset-controller

e2e-worker-w4-a48099-openshift-default-577864759

SuccessfulCreate

Created pod: e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

AddedInterface

Add eth0 [10.133.1.41/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

Created

Container created

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

AddedInterface

Add eth0 [10.134.0.141/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

AddedInterface

Add eth0 [10.132.0.128/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

AddedInterface

Add eth0 [10.133.1.40/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

Created

Container created

openshift-ingress

multus

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

AddedInterface

Add eth0 [10.132.0.127/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-qpjb7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-5bms2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-j8mfj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-trkbg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-fchpl

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

ProbeError

Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp); attempting to pull the image may not succeed.
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

Unhealthy

Startup probe failed: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

ProbeError

Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

Unhealthy

Startup probe failed: Get "http://10.133.1.41:15021/healthz/ready": dial tcp 10.133.1.41:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w4-a48099-openshift-default-dockercfg-8mhxp); attempting to pull the image may not succeed.
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-nkc44

Unhealthy

Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

ProbeError

Startup probe error: Get "http://10.133.1.41:15021/healthz/ready": dial tcp 10.133.1.41:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

ProbeError

Startup probe error: Get "http://10.133.1.40:15021/healthz/ready": dial tcp 10.133.1.40:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-vf6t5

Unhealthy

Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-cdk5z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

ProbeError

Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-gk2pd

Unhealthy

Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-a48099-openshift-default-577864759-6x4ct

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-1c4191-84cdfb95f7-qfl28

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-1c4191-65f847c49f-dnq8g

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-b-w0-1c4191-openshift-default-7d749f944c-9fsfm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

Killing

Stopping container payload-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-9gnnl

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-mpdzz
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-58d898bd55 from 6 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-mpdzz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7d1ec3-5c5c8b5577-5z46v

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7d1ec3-6b66456595-2fczg

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7d1ec3-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w0-7d1ec3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c from 1 to 3

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

openshift-ingress

replicaset-controller

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c

SuccessfulCreate

Created pod: e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

AddedInterface

Add eth0 [10.133.1.46/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

AddedInterface

Add eth0 [10.134.0.142/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w0-7d1ec3-openshift-default-dockercfg-thhxl); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w0-7d1ec3-openshift-default-dockercfg-thhxl); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

Unhealthy

Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

Unhealthy

Startup probe failed: Get "http://10.133.1.46:15021/healthz/ready": dial tcp 10.133.1.46:15021: connect: connection refused
(x3)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-kdkzl

ProbeError

Startup probe error: Get "http://10.133.1.46:15021/healthz/ready": dial tcp 10.133.1.46:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-9npk6

ProbeError

Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-7d1ec3-openshift-default-86c7d47c6c-5nmgn

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

opendatahub

multus

test-curl-7443-0b0e22

AddedInterface

Add eth0 [10.133.1.49/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7443-0b0e22

Started

Container started

opendatahub

kubelet

test-curl-7443-0b0e22

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7443-0b0e22

Created

Container created

opendatahub

kubelet

test-curl-7443-0b0e22

Killing

Stopping container test-curl-7443-0b0e22

opendatahub

kubelet

test-curl-7443-5cc14f

Started

Container started

opendatahub

multus

test-curl-7443-5cc14f

AddedInterface

Add eth0 [10.133.1.50/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7443-5cc14f

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7443-5cc14f

Created

Container created

opendatahub

kubelet

test-curl-7443-5cc14f

Killing

Stopping container test-curl-7443-5cc14f

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-58d898bd55 from 4 to 3

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ztp4p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-a-w1-897c87-openshift-default-6dd89bc94f-xttgv

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-897c87-96bc99f-jr2x5

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-897c87-57666cd899-bjrv7

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-b-w1-fd2672-openshift-default-54dd6ccd69-kv9n5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-fd2672-7d58df9c59-2glzq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-fd2672-659f849fc9-pqzgj

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-a-a12c35-openshift-default-84d7454997-whr8l

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-a12c35-6dc5c85cff-wbz7d

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-a12c35-6cdb45475c-brsf2

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-2fd4c9-openshift-default-69b7b9b68f-xdmp8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2fd4c9-7b49b85457-4pklk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2fd4c9-5d6bc9cdff-j8c2r

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xsdd7

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ait-3951eb77-openshift-default-85859ff58c

SuccessfulCreate

Created pod: e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

openshift-ingress

gateway_labeler_controller

e2e-ait-3951eb77

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-3951eb77

openshift-ingress

deployment-controller

e2e-ait-3951eb77-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-3951eb77-openshift-default-85859ff58c from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-58d898bd55

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-58d898bd55-xsdd7

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-58d898bd55 from 3 to 2

openshift-ingress

multus

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

AddedInterface

Add eth0 [10.134.0.143/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-3951eb77

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-3951eb77-77c58c7955 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-3951eb77-77c58c7955

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-3951eb77-77c58c7955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-3951eb77: serviceaccount "payload-processing-e2e-ait-3951eb77" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-3951eb77-558dd68b9c

FailedCreate

Error creating: pods "payload-processing-e2e-ait-3951eb77-558dd68b9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-3951eb77: serviceaccount "payload-processing-e2e-ait-3951eb77" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-3951eb77-558dd68b9c

SuccessfulCreate

Created pod: payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-3951eb77-77c58c7955

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7

openshift-ingress

deployment-controller

payload-processing-e2e-ait-3951eb77

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-3951eb77-558dd68b9c from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7

Created

Container created

openshift-ingress

multus

payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh

AddedInterface

Add eth0 [10.134.0.145/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7

AddedInterface

Add eth0 [10.134.0.144/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

Unhealthy

Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

ProbeError

Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-3951eb77-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-3951eb77-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-ait-3951eb77-558dd68b9c-2dzqh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3951eb77-77c58c7955-8n2p7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-3951eb77-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-3951eb77-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-ait-3951eb77-openshift-default-85859ff58c

FailedCreate

Error creating: pods "e2e-ait-3951eb77-openshift-default-85859ff58c-" is forbidden: error looking up service account openshift-ingress/e2e-ait-3951eb77-openshift-default: serviceaccount "e2e-ait-3951eb77-openshift-default" not found

openshift-ingress

deployment-controller

e2e-ait-3951eb77-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-3951eb77-openshift-default-85859ff58c from 0 to 1

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-3951eb77-openshift-default-85859ff58c-gf998

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

opendatahub

kubelet

maas-controller-74b5bc6fff-vqr9s

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-74b5bc6fff

SuccessfulDelete

Deleted pod: maas-controller-74b5bc6fff-vqr9s

opendatahub

replicaset-controller

maas-controller-74b5bc6fff

SuccessfulCreate

Created pod: maas-controller-74b5bc6fff-gmvbm

opendatahub

multus

maas-controller-74b5bc6fff-gmvbm

AddedInterface

Add eth0 [10.134.0.147/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-74b5bc6fff-gmvbm

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19"

opendatahub

kubelet

maas-controller-74b5bc6fff-gmvbm

Started

Container started

opendatahub

kubelet

maas-controller-74b5bc6fff-gmvbm

Created

Container created

opendatahub

kubelet

maas-controller-74b5bc6fff-gmvbm

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19" in 1.917s (1.917s including waiting). Image size: 200094741 bytes.

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-58d898bd55 from 2 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-sd9zb

Killing

Stopping container istio-proxy

opendatahub

maas-controller-74b5bc6fff-gmvbm_b99db56c-4ec8-48f2-ac9d-6685209628b5

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-74b5bc6fff-gmvbm_b99db56c-4ec8-48f2-ac9d-6685209628b5 became leader
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-58d898bd55 from 1 to 4
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-c6zlb

AddedInterface

Add eth0 [10.134.0.149/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-w2nmc

AddedInterface

Add eth0 [10.134.0.148/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-b4bkr

AddedInterface

Add eth0 [10.133.1.56/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

Unhealthy

Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

Unhealthy

Startup probe failed: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

ProbeError

Startup probe error: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

ProbeError

Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

ProbeError

Startup probe error: Get "http://10.133.1.56:15021/healthz/ready": dial tcp 10.133.1.56:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

Unhealthy

Startup probe failed: Get "http://10.133.1.56:15021/healthz/ready": dial tcp 10.133.1.56:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

ProbeError

Startup probe error: Get "http://10.133.1.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

Unhealthy

Startup probe failed: Get "http://10.133.1.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

Unhealthy

Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

ProbeError

Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

Unhealthy

Startup probe failed: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

ProbeError

Startup probe error: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-74b5bc6fff from 1 to 0

opendatahub

replicaset-controller

maas-controller-74b5bc6fff

SuccessfulDelete

Deleted pod: maas-controller-74b5bc6fff-gmvbm

opendatahub

kubelet

maas-controller-74b5bc6fff-gmvbm

Killing

Stopping container manager
(x3)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-74b5bc6fff from 0 to 1

opendatahub

replicaset-controller

maas-controller-74b5bc6fff

SuccessfulCreate

Created pod: maas-controller-74b5bc6fff-6xcrb

opendatahub

multus

maas-controller-74b5bc6fff-6xcrb

AddedInterface

Add eth0 [10.134.0.150/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-74b5bc6fff-6xcrb

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19"

opendatahub

kubelet

maas-controller-74b5bc6fff-6xcrb

Started

Container started

opendatahub

kubelet

maas-controller-74b5bc6fff-6xcrb

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:7b5499413cc752a539948674e1c091771cbbfa775cf0e6b7a1c1fa2b31d94f19" in 795ms (795ms including waiting). Image size: 200094741 bytes.

opendatahub

kubelet

maas-controller-74b5bc6fff-6xcrb

Created

Container created
(x2)

opendatahub

kubelet

ai-gateway-controller-559cb95fbf-czdwv

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"
(x2)

opendatahub

kubelet

ai-gateway-controller-559cb95fbf-czdwv

Started

Container started

opendatahub

kubelet

ai-gateway-controller-559cb95fbf-czdwv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 913ms (913ms including waiting). Image size: 183979433 bytes.
(x2)

opendatahub

kubelet

ai-gateway-controller-559cb95fbf-czdwv

Created

Container created

opendatahub

maas-controller-74b5bc6fff-6xcrb_3eda95f4-8d3c-4270-bf86-9426ebb61193

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-74b5bc6fff-6xcrb_3eda95f4-8d3c-4270-bf86-9426ebb61193 became leader

opendatahub

ai-gateway-controller-559cb95fbf-czdwv_4b7e679d-2973-4e53-a464-e5f00a8d0e51

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-559cb95fbf-czdwv_4b7e679d-2973-4e53-a464-e5f00a8d0e51 became leader

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-cd9pn

AddedInterface

Add eth0 [10.134.0.151/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-58d898bd55 from 4 to 8

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-8ng9q

AddedInterface

Add eth0 [10.133.1.59/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-kb7j2

AddedInterface

Add eth0 [10.133.1.60/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-xnxx8

AddedInterface

Add eth0 [10.132.0.129/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

Created

Container created
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

ProbeError

Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

Unhealthy

Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

ProbeError

Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

Unhealthy

Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

ProbeError

Startup probe error: Get "http://10.133.1.60:15021/healthz/ready": dial tcp 10.133.1.60:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

ProbeError

Startup probe error: Get "http://10.133.1.59:15021/healthz/ready": dial tcp 10.133.1.59:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

Unhealthy

Startup probe failed: Get "http://10.133.1.60:15021/healthz/ready": dial tcp 10.133.1.60:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

Unhealthy

Startup probe failed: Get "http://10.133.1.59:15021/healthz/ready": dial tcp 10.133.1.59:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

ProbeError

Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

Unhealthy

Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

Unhealthy

Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

ProbeError

Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

ProbeError

Startup probe error: Get "http://10.133.1.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

Unhealthy

Startup probe failed: Get "http://10.133.1.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

ProbeError

Startup probe error: Get "http://10.133.1.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

Unhealthy

Startup probe failed: Get "http://10.133.1.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-tgfm2

AddedInterface

Add eth0 [10.134.0.152/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason:
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-58d898bd55 from 8 to 10

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-58d898bd55-p22bd

AddedInterface

Add eth0 [10.132.0.130/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

Started

Container started
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

Unhealthy

Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

ProbeError

Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

ProbeError

Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

Unhealthy

Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

ProbeError

Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

Unhealthy

Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

Unhealthy

Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

ProbeError

Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.ClusterRoleBinding /llm-e2e-prio-pool-25304531-epp-auth-rb

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.ServiceAccount llm/e2e-prio-pool-25304531-epp-sa

llm

deployment-controller

e2e-prio-pool-25304531-kserve

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-25304531-kserve-5548ccf5c4 from 0 to 1

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.Secret llm/e2e-prio-pool-25304531-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.Deployment llm/e2e-prio-pool-25304531-kserve

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.Role llm/e2e-prio-pool-25304531-epp-role

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.Service llm/e2e-prio-pool-25304531-kserve-workload-svc

llm

replicaset-controller

e2e-prio-pool-25304531-kserve-5548ccf5c4

SuccessfulCreate

Created pod: e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8

llm

multus

e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8

AddedInterface

Add eth0 [10.133.1.64/23] from ovn-kubernetes

llm

kubelet

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

Pulling

Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827"

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.Service llm/e2e-prio-pool-25304531-epp-service

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.Deployment llm/e2e-prio-pool-25304531-kserve-router-scheduler

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

Created v1.RoleBinding llm/e2e-prio-pool-25304531-epp-rb

llm

deployment-controller

e2e-prio-pool-25304531-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f from 0 to 1

llm

replicaset-controller

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f

SuccessfulCreate

Created pod: e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

llm

multus

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

AddedInterface

Add eth0 [10.134.0.153/23] from ovn-kubernetes

llm

kubelet

e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8

Started

Container started

llm

kubelet

e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8

Created

Container created

llm

kubelet

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

Created

Container created

llm

kubelet

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827" in 1.916s (1.916s including waiting). Image size: 184206420 bytes.

llm

kubelet

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

Started

Container started

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Updated

Updated v1.Secret llm/e2e-prio-pool-25304531-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Updated

Updated v1.HTTPRoute llm/e2e-prio-pool-25304531-kserve-route
(x9)

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Created

(combined from similar events): Created v1.DestinationRule llm/e2e-prio-pool-25304531-kserve-shadow-svc

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

LLMInferenceServiceReady

LLMInferenceService [e2e-prio-pool-25304531] is Ready

llm

LLMInferenceServiceController

e2e-prio-pool-25304531

Deleted

Deleted v1.ClusterRoleBinding /llm-e2e-prio-pool-25304531-epp-auth-rb

llm

kubelet

e2e-prio-pool-25304531-kserve-5548ccf5c4-zzxc8

Killing

Stopping container main

llm

kubelet

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

Killing

Stopping container main

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.Deployment llm/e2e-same-id-a-f6cbc132-kserve

llm

replicaset-controller

e2e-same-id-a-f6cbc132-kserve-5889cd8685

SuccessfulCreate

Created pod: e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.DestinationRule llm/e2e-same-id-a-f6cbc132-kserve-workload-svc

llm

deployment-controller

e2e-same-id-a-f6cbc132-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-a-f6cbc132-kserve-5889cd8685 from 0 to 1

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.Secret llm/e2e-same-id-a-f6cbc132-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.Service llm/e2e-same-id-a-f6cbc132-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.HTTPRoute llm/e2e-same-id-a-f6cbc132-kserve-route

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.NetworkPolicy llm/e2e-same-id-a-f6cbc132-prometheus-scraping

llm

multus

e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh

AddedInterface

Add eth0 [10.133.1.67/23] from ovn-kubernetes

llm

kubelet

e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh

Started

Container started

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.PodMonitor llm/e2e-same-id-a-f6cbc132-kserve-llmisvc-engine-default

llm

kubelet

e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh

Created

Container created

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Created

Created v1.PodMonitor llm/e2e-same-id-a-f6cbc132-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

Updated

Updated v1.Secret llm/e2e-same-id-a-f6cbc132-kserve-self-signed-certs
(x4)

llm

kubelet

e2e-prio-pool-25304531-kserve-router-scheduler-5664d6b58f-jvsk9

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-a-f6cbc132] is Ready

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.Secret llm/e2e-same-id-b-f6cbc132-kserve-self-signed-certs

llm

kubelet

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

Created

Container created

llm

kubelet

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

Started

Container started

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.HTTPRoute llm/e2e-same-id-b-f6cbc132-kserve-route

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.DestinationRule llm/e2e-same-id-b-f6cbc132-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.Service llm/e2e-same-id-b-f6cbc132-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.PodMonitor llm/e2e-same-id-b-f6cbc132-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.PodMonitor llm/e2e-same-id-b-f6cbc132-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.Deployment llm/e2e-same-id-b-f6cbc132-kserve

llm

replicaset-controller

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c

SuccessfulCreate

Created pod: e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

llm

deployment-controller

e2e-same-id-b-f6cbc132-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-b-f6cbc132-kserve-6948cd4f9c from 0 to 1

llm

multus

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

AddedInterface

Add eth0 [10.134.0.154/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Created

Created v1.NetworkPolicy llm/e2e-same-id-b-f6cbc132-prometheus-scraping

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

Updated

Updated v1.Secret llm/e2e-same-id-b-f6cbc132-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-b-f6cbc132] is Ready
(x3)

llm

LLMInferenceServiceController

e2e-same-id-a-f6cbc132

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-b-f6cbc132 in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-a-f6cbc132; safe to ignore if traffic splitting is planned
(x5)

llm

LLMInferenceServiceController

e2e-same-id-b-f6cbc132

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-a-f6cbc132 in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-b-f6cbc132; safe to ignore if traffic splitting is planned

llm

kubelet

e2e-same-id-a-f6cbc132-kserve-5889cd8685-78cbh

Killing

Stopping container main

llm

kubelet

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

Killing

Stopping container main

llm

kubelet

e2e-same-id-b-f6cbc132-kserve-6948cd4f9c-zlqx8

Unhealthy

Readiness probe failed: Get "https://10.134.0.154:8000/ready": dial tcp 10.134.0.154:8000: connect: connection refused

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-5fb6d674c6 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-5fb6d674c6

SuccessfulCreate

Created pod: payload-pre-processing-5fb6d674c6-8c4kb

openshift-ingress

multus

payload-pre-processing-5fb6d674c6-8c4kb

AddedInterface

Add eth0 [10.133.1.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-5fb6d674c6-8c4kb

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-5fb6d674c6-8c4kb

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-5fb6d674c6-8c4kb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-j678p

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-7b98bb678c from 1 to 0

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulDelete

Deleted pod: payload-pre-processing-7b98bb678c-j678p

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-6dbdbcbf6f from 0 to 1

openshift-ingress

multus

payload-processing-6dbdbcbf6f-6dq5z

AddedInterface

Add eth0 [10.133.1.73/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-6dbdbcbf6f-6dq5z

Started

Container started

openshift-ingress

kubelet

payload-processing-6dbdbcbf6f-6dq5z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-6dbdbcbf6f

SuccessfulCreate

Created pod: payload-processing-6dbdbcbf6f-6dq5z

openshift-ingress

kubelet

payload-processing-6dbdbcbf6f-6dq5z

Created

Container created

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-55fbbbf9dc from 1 to 0

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulDelete

Deleted pod: payload-processing-55fbbbf9dc-rfb94

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rfb94

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-55775b4dc9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-b6ff555c6

SuccessfulCreate

Created pod: payload-processing-b6ff555c6-lxd69

openshift-ingress

replicaset-controller

payload-pre-processing-55775b4dc9

SuccessfulCreate

Created pod: payload-pre-processing-55775b4dc9-fvzqg

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-b6ff555c6 from 0 to 1

openshift-ingress

multus

payload-processing-b6ff555c6-lxd69

AddedInterface

Add eth0 [10.133.1.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-55775b4dc9-fvzqg

Started

Container started

openshift-ingress

kubelet

payload-processing-b6ff555c6-lxd69

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-b6ff555c6-lxd69

Created

Container created

openshift-ingress

kubelet

payload-processing-b6ff555c6-lxd69

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-55775b4dc9-fvzqg

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-55775b4dc9-fvzqg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-55775b4dc9-fvzqg

AddedInterface

Add eth0 [10.133.1.75/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-6dbdbcbf6f from 1 to 0

openshift-ingress

kubelet

payload-processing-6dbdbcbf6f-6dq5z

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-5fb6d674c6

SuccessfulDelete

Deleted pod: payload-pre-processing-5fb6d674c6-8c4kb

openshift-ingress

kubelet

payload-pre-processing-5fb6d674c6-8c4kb

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-6dbdbcbf6f

SuccessfulDelete

Deleted pod: payload-processing-6dbdbcbf6f-6dq5z

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-5fb6d674c6 from 1 to 0

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5bcd7997 from 0 to 1
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

FailedCreate

Error creating: pods "payload-processing-external-model-5bcd7997-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

multus

payload-processing-external-model-5bcd7997-knxw6

AddedInterface

Add eth0 [10.133.1.76/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

SuccessfulCreate

Created pod: payload-processing-external-model-5bcd7997-knxw6

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-knxw6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-knxw6

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-knxw6

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-knxw6

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-worker-single-main-42db1c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-42db1c

openshift-ingress

deployment-controller

e2e-worker-single-main-42db1c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd9 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd9

SuccessfulCreate

Created pod: e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

openshift-ingress

multus

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

AddedInterface

Add eth0 [10.133.1.90/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-42db1c-85c47756c5

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-42db1c-85c47756c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-42db1c: serviceaccount "payload-processing-e2e-worker-single-main-42db1c" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-42db1c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-42db1c-85c47756c5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-42db1c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-42db1c-569dbc7659 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-42db1c-569dbc7659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-42db1c-569dbc7659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-42db1c: serviceaccount "payload-processing-e2e-worker-single-main-42db1c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-42db1c-569dbc7659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-42db1c-85c47756c5

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

AddedInterface

Add eth0 [10.133.1.91/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

AddedInterface

Add eth0 [10.134.0.156/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

ProbeError

Startup probe error: Get "http://10.133.1.90:15021/healthz/ready": dial tcp 10.133.1.90:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

Unhealthy

Startup probe failed: Get "http://10.133.1.90:15021/healthz/ready": dial tcp 10.133.1.90:15021: connect: connection refused
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-42db1c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-42db1c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-42db1c-569dbrhmgl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-42db1c-85c47756chhgcq

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.156:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-single-main-42db1c-openshift-default-6bd6c77dd98qhcw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

Unhealthy

Readiness probe failed: Get "http://10.133.1.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-8ng9q

ProbeError

Readiness probe error: Get "http://10.133.1.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-678d87766f from 1 to 0

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-kfkl9

Killing

Stopping container manager

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-678d87766f-kfkl9

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-rcf8t
(x2)

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

NeedsReinstall

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

ComponentUnhealthy

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rcf8t

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rcf8t

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-rcf8t

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-rcf8t

AddedInterface

Add eth0 [10.132.0.131/23] from ovn-kubernetes
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install
(x3)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-rcf8t_5dbfa046-35a1-4c20-b56c-7bc72edb583d

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-rcf8t_5dbfa046-35a1-4c20-b56c-7bc72edb583d became leader
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

ProbeError

Readiness probe error: Get "http://10.133.1.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-b4bkr

Unhealthy

Readiness probe failed: Get "http://10.133.1.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

Unhealthy

Readiness probe failed: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

Unhealthy

Readiness probe failed: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-c6zlb

ProbeError

Readiness probe error: Get "http://10.134.0.149:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

ProbeError

Readiness probe error: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-xnxx8

Unhealthy

Readiness probe failed: Get "http://10.132.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

ProbeError

Readiness probe error: Get "http://10.134.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-cd9pn

Unhealthy

Readiness probe failed: Get "http://10.134.0.151:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-w2nmc

ProbeError

Readiness probe error: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

ProbeError

Readiness probe error: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-ck46t

Unhealthy

Readiness probe failed: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

Unhealthy

Readiness probe failed: Get "http://10.133.1.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-kb7j2

ProbeError

Readiness probe error: Get "http://10.133.1.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

ProbeError

Readiness probe error: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

Unhealthy

Readiness probe failed: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-p22bd

ProbeError

Readiness probe error: Get "http://10.132.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-58d898bd55-tgfm2

Unhealthy

Readiness probe failed: Get "http://10.134.0.152:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)