Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm to ip-10-0-135-4.us-west-1.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-k45wc

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-k45wc to ip-10-0-135-4.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-c62113

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-c62113: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-27bb49

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-27bb49: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-c0ab5f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-c0ab5f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-c2cde5

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-c2cde5: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-7af38e

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-7af38e: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w1-795045

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w1-795045: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-82706f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-82706f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-fa0149

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-fa0149: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-f4d202

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-f4d202: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-8bf4k

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-8bf4k to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8wpzd to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-58c98c4485dftg7

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-58c98c4485dftg7 to ip-10-0-129-138.us-west-1.compute.internal

opendatahub

maas-controller-8677bb4cb4-5d7s9

Scheduled

Successfully assigned opendatahub/maas-controller-8677bb4cb4-5d7s9 to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

model-serving-api-75bd89fdfc-lgdkj

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-lgdkj to ip-10-0-129-138.us-west-1.compute.internal

opendatahub

odh-model-controller-769875db56-l5vpc

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-l5vpc to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hs2x2 to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-hw8r9 to ip-10-0-129-138.us-west-1.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-h7sl2

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-h7sl2 to ip-10-0-129-138.us-west-1.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8 to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-jjhm5 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-kdtlh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kdtlh to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-kh4xr to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs to ip-10-0-129-138.us-west-1.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-jjl7l

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-jjl7l to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-m59rh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m59rh to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-m9d6q to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-84fbf4d7ff-zlfjk

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-84fbf4d7ff-zlfjk to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-nbx74

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nbx74 to ip-10-0-129-138.us-west-1.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-kn75t

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-kn75t to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-nwszr

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-nwszr to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-p7nkd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-p7nkd to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qnxqv to ip-10-0-129-138.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-tkbx8

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-tkbx8 to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

test-curl-7117-3e8e8b

Scheduled

Successfully assigned opendatahub/test-curl-7117-3e8e8b to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-gstds

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-gstds to ip-10-0-140-18.us-west-1.compute.internal

kuadrant-system

authorino-56c6c9ddd5-bbc54

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-bbc54 to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

test-curl-7117-92f403

Scheduled

Successfully assigned opendatahub/test-curl-7117-92f403 to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4qznm to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-84fbf4d7ff-8vkgx

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-84fbf4d7ff-8vkgx to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt to ip-10-0-140-18.us-west-1.compute.internal

kuadrant-system

authorino-57f7f8659f-jh68b

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-jh68b to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9 to ip-10-0-129-138.us-west-1.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-wbtv2

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-wbtv2 to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh to ip-10-0-129-138.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-7565df5f-548hr

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-548hr to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6 to ip-10-0-140-18.us-west-1.compute.internal

kuadrant-system

authorino-65457c7d56-kdp2h

Scheduled

Successfully assigned kuadrant-system/authorino-65457c7d56-kdp2h to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4 to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-q6hlp

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-q6hlp to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-qpptj

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-qpptj to ip-10-0-140-18.us-west-1.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-z22gn

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-z22gn to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-s8k5g to ip-10-0-129-138.us-west-1.compute.internal

kuadrant-system

authorino-659cdbc977-49rwz

Scheduled

Successfully assigned kuadrant-system/authorino-659cdbc977-49rwz to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-processing-7bb9489676-tv42d

Scheduled

Successfully assigned openshift-ingress/payload-processing-7bb9489676-tv42d to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vkm88 to ip-10-0-140-18.us-west-1.compute.internal

kuadrant-system

authorino-6f6944bcdd-lwm7l

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-lwm7l to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8 to ip-10-0-140-18.us-west-1.compute.internal

kuadrant-system

authorino-9bdb54dbc-72v2t

Scheduled

Successfully assigned kuadrant-system/authorino-9bdb54dbc-72v2t to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6 to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5 to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh to ip-10-0-129-138.us-west-1.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-9x6jf

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-9x6jf to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-gjl74

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gjl74 to ip-10-0-135-4.us-west-1.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-xgzhk

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-xgzhk to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw to ip-10-0-129-138.us-west-1.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-6lccj

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-6lccj to ip-10-0-129-138.us-west-1.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc to ip-10-0-135-4.us-west-1.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-gzdl5

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-gzdl5 to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

maas-controller-79c79ff55f-qc5qq

Scheduled

Successfully assigned opendatahub/maas-controller-79c79ff55f-qc5qq to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-xbf55

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbf55 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67 to ip-10-0-129-138.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-7565df5f-spt9h

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-spt9h to ip-10-0-135-4.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-xml2m

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xml2m to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

router-default-54848578cf-gq6dc

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-54848578cf-gq6dc

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-54848578cf-gq6dc

Scheduled

Successfully assigned openshift-ingress/router-default-54848578cf-gq6dc to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k to ip-10-0-129-138.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-06099f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-06099f: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59 to ip-10-0-135-4.us-west-1.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-hvhhm

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-hvhhm to ip-10-0-135-4.us-west-1.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-kj776

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-kj776 to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-5x8z8 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq to ip-10-0-140-18.us-west-1.compute.internal

openshift-jobset-operator

jobset-controller-manager-9db885d67-lfq7q

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-9db885d67-lfq7q to ip-10-0-135-4.us-west-1.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6 to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-75b555675f-xx482

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-75b555675f-xx482 to ip-10-0-129-138.us-west-1.compute.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-8ndrn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-8ndrn to ip-10-0-129-138.us-west-1.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-jt25s

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-jt25s to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5 to ip-10-0-135-4.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-5497dccdcc-bbs4k

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5497dccdcc-bbs4k to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6 to ip-10-0-129-138.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-586b974f6-dt8ql

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-586b974f6-dt8ql to ip-10-0-129-138.us-west-1.compute.internal

openshift-ingress

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz to ip-10-0-129-138.us-west-1.compute.internal

ai-tenants

e2e-extmod-a-79badd

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-79badd: MaasTenantConfig is not available to restore revocation operands

kuadrant-system

limitador-limitador-7c9b87d574-mzk5s

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-mzk5s to ip-10-0-129-138.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-8ffbc55d8-rqd7j

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-8ffbc55d8-rqd7j to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

ai-gateway-controller-5d8dddb676-qkgcd

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-qkgcd to ip-10-0-129-138.us-west-1.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-jk4fq

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-jk4fq to ip-10-0-140-18.us-west-1.compute.internal

opendatahub

ai-gateway-controller-6d4cf6d5f-9wgr6

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-6d4cf6d5f-9wgr6 to ip-10-0-129-138.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-7565df5f-vwrdz

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-7565df5f-vwrdz to ip-10-0-140-18.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-cjjc7

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-cjjc7 to ip-10-0-140-18.us-west-1.compute.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-54848578cf from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-54848578cf

SuccessfulCreate

Created pod: router-default-54848578cf-gq6dc
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-54848578cf-gq6dc

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-54848578cf-gq6dc

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x4)

openshift-ingress

kubelet

router-default-54848578cf-gq6dc

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-54848578cf-gq6dc

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-54848578cf-gq6dc

AddedInterface

Add eth0 [10.132.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-54848578cf-gq6dc

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

router-default-54848578cf-gq6dc

Started

Container started

openshift-ingress

kubelet

router-default-54848578cf-gq6dc

Created

Container created

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-q6hlp

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-q6hlp

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-q6hlp

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-q6hlp

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-q6hlp

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-q6hlp

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.11s (4.11s including waiting). Image size: 185476477 bytes.
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-q6hlp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-q6hlp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-vkm88

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-75b555675f

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-75b555675f-xx482

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-vkm88

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-75b555675f from 0 to 1

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-778b7b5c47 from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-75b555675f-xx482

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-75b555675f-xx482

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.062s (4.062s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-75b555675f-xx482

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 7.224s (7.224s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-75b555675f-xx482

Started

Container started

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-75b555675f-xx482

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-7565df5f

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-7565df5f-vwrdz

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-7565df5f from 0 to 3

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-7565df5f

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-7565df5f-spt9h

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-7565df5f

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-7565df5f-548hr

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-7565df5f-vwrdz

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-7565df5f-548hr

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-7565df5f-spt9h

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"
(x7)

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

FailedCreate

Error creating: pods "cert-manager-webhook-5dbbbf75cc-" is forbidden: error looking up service account cert-manager/cert-manager-webhook: serviceaccount "cert-manager-webhook" not found
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-jjl7l

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-jjl7l

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

opendatahub-operator-system

operator-lifecycle-manager

install-w4lfs

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-jjl7l

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked
(x2)

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.375s (3.375s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.392s (4.392s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.482s (3.482s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 822ms (822ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-548hr

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 797ms (797ms including waiting). Image size: 273709574 bytes.
(x11)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-jjl7l

Created

Container created

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-jjl7l

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Started

Container started

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Started

Container started

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-vwrdz

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 803ms (803ms including waiting). Image size: 273709574 bytes.

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-jjl7l

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 4.104s (4.104s including waiting). Image size: 309650901 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-7565df5f-spt9h

Created

Container created

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-h7sl2

opendatahub-operator-system

opendatahub-operator-controller-manager-7565df5f-548hr_0943c2e3-0ea0-4197-802c-055f7468924c

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-7565df5f-548hr_0943c2e3-0ea0-4197-802c-055f7468924c became leader

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-h7sl2

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-h7sl2

Created

Container created

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-h7sl2

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-h7sl2

Started

Container started

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-kj776

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-6lccj

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-kj776

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-kj776

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-6lccj

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-6lccj

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-8bf4k

cert-manager

kubelet

cert-manager-597b95cb84-8bf4k

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

multus

cert-manager-597b95cb84-8bf4k

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-kj776

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 3.473s (3.473s including waiting). Image size: 190804149 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-kj776

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-kj776

Started

Container started

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-6lccj

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 3.023s (3.023s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-6lccj

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-6lccj

Created

Container created

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-9x6jf

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

dns-operator-controller-manager-8557cbdb84-kj776_f6861fc8-bc06-4d28-a99b-7fba4b4843a0

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-kj776_f6861fc8-bc06-4d28-a99b-7fba4b4843a0 became leader

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

cert-manager

kubelet

cert-manager-597b95cb84-8bf4k

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-6lccj_11747d76-e5c2-4d4c-aa0a-a2521125fe5b became leader

kuadrant-system

multus

authorino-operator-7b7bb4bc77-9x6jf

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-8bf4k

Created

Container created

cert-manager

kubelet

cert-manager-597b95cb84-8bf4k

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.176s (3.176s including waiting). Image size: 309650901 bytes.

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-9x6jf

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

kuadrant-system

authorino-operator-7b7bb4bc77-9x6jf_76927037-d5ac-4183-9e3c-3238bcb202a9

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-9x6jf_76927037-d5ac-4183-9e3c-3238bcb202a9 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-9x6jf

Started

Container started

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-9x6jf

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-9x6jf

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 2.523s (2.523s including waiting). Image size: 167738465 bytes.

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-cjjc7

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-cjjc7

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-cjjc7

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-jt25s

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-jt25s

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-jt25s

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-cjjc7

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-cjjc7

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 4.915s (4.915s including waiting). Image size: 424244516 bytes.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-cjjc7

Created

Container created

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-cjjc7_f65477bc-3bb7-43fd-aa9c-f57ad5cecaeb

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-cjjc7_f65477bc-3bb7-43fd-aa9c-f57ad5cecaeb became leader

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-jt25s

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 3.484s (3.484s including waiting). Image size: 177967631 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-jt25s

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-jt25s

Started

Container started

kuadrant-system

limitador-operator-controller-manager-5778bb4449-jt25s_5e58526a-4774-4170-a690-9a42cca746ac

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-jt25s_5e58526a-4774-4170-a690-9a42cca746ac became leader

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-k45wc

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k45wc

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-k45wc

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-b66g7"

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k45wc

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k45wc

Created

Container created

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-84fbf4d7ff from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-84fbf4d7ff

SuccessfulCreate

Created pod: kube-auth-proxy-84fbf4d7ff-8vkgx

openshift-ingress

replicaset-controller

kube-auth-proxy-84fbf4d7ff

SuccessfulCreate

Created pod: kube-auth-proxy-84fbf4d7ff-zlfjk

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-gvntn"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k45wc

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 5.742s (5.742s including waiting). Image size: 342538866 bytes.

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-58c98c4485

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-58c98c4485dftg7

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-58c98c4485dftg7

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-zlfjk

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-84fbf4d7ff-zlfjk

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-8vkgx

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-84fbf4d7ff-8vkgx

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-58c98c4485dftg7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-58c98c4485 from 0 to 1

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-58c98c4485dftg7

Created

Container created

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-58c98c4485dftg7

Started

Container started

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-jk4fq

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-zlfjk

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-9db885d67-lfq7q

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-9db885d67

SuccessfulCreate

Created pod: jobset-controller-manager-9db885d67-lfq7q

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-zlfjk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.784s (2.784s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-zlfjk

Started

Container started

openshift-jobset-operator

multus

jobset-controller-manager-9db885d67-lfq7q

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

kuadrant-system

multus

limitador-limitador-74dfcfb894-jk4fq

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jk4fq

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-8vkgx

Started

Container started

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-9db885d67 from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-8vkgx

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-84fbf4d7ff-8vkgx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.912s (2.912s including waiting). Image size: 188904677 bytes.

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

kuadrant-system

multus

authorino-57f7f8659f-jh68b

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-57f7f8659f-jh68b

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-jh68b

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-mzk5s

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-bbc54

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jk4fq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 2.905s (2.905s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jk4fq

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jk4fq

Started

Container started

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-mzk5s

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

kubelet

authorino-56c6c9ddd5-bbc54

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-56c6c9ddd5-bbc54

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

kuadrant-system

multus

limitador-limitador-7c9b87d574-mzk5s

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-mzk5s

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-jh68b

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-bbc54

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.248s (2.248s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-bbc54

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-bbc54

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-jh68b

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.838s (3.838s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-57f7f8659f-jh68b

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-9db885d67-lfq7q

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 4.597s (4.597s including waiting). Image size: 182184678 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-mzk5s

Started

Container started

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-mzk5s

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 1.631s (1.631s including waiting). Image size: 133379307 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-9db885d67-lfq7q

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-9db885d67-lfq7q

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

openshift-jobset-operator

jobset-controller-manager-9db885d67-lfq7q_0ab0f4b0-e42f-414d-b853-0210a60a73fa

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-9db885d67-lfq7q_0ab0f4b0-e42f-414d-b853-0210a60a73fa became leader

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-jh68b

kuadrant-system

kubelet

authorino-57f7f8659f-jh68b

Killing

Stopping container authorino

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-jk4fq

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-jk4fq

Killing

Stopping container limitador

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-wbtv2

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-wbtv2

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-wbtv2

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-wbtv2

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 3.114s (3.114s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-wbtv2

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-wbtv2

Started

Container started

opendatahub

kserve-module-controller-manager-7566cf5fc8-wbtv2_1fb8711e-64d4-4c7f-999e-846824cfc159

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-wbtv2_1fb8711e-64d4-4c7f-999e-846824cfc159 became leader

opendatahub

replicaset-controller

maas-controller-79c79ff55f

SuccessfulCreate

Created pod: maas-controller-79c79ff55f-qc5qq

opendatahub

replicaset-controller

maas-controller-8677bb4cb4

SuccessfulCreate

Created pod: maas-controller-8677bb4cb4-5d7s9

opendatahub

kubelet

maas-controller-79c79ff55f-qc5qq

FailedMount

MountVolume.SetUp failed for volume "webhook-cert" : secret "maas-controller-webhook-cert" not found

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-79c79ff55f from 0 to 1

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-8677bb4cb4 from 0 to 1

opendatahub

multus

maas-controller-8677bb4cb4-5d7s9

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

opendatahub

multus

maas-controller-79c79ff55f-qc5qq

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-79c79ff55f-qc5qq

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:36c8291eb6277ee16bcd2bc119baac75c285cfe906a702d29217a2100731824b"

opendatahub

kubelet

maas-controller-79c79ff55f-qc5qq

Started

Container started

opendatahub

kubelet

maas-controller-79c79ff55f-qc5qq

Created

Container created

opendatahub

kubelet

maas-controller-79c79ff55f-qc5qq

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:36c8291eb6277ee16bcd2bc119baac75c285cfe906a702d29217a2100731824b" in 5.658s (5.658s including waiting). Image size: 200141006 bytes.

opendatahub

kubelet

maas-controller-8677bb4cb4-5d7s9

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:36c8291eb6277ee16bcd2bc119baac75c285cfe906a702d29217a2100731824b" in 5.699s (5.699s including waiting). Image size: 200141006 bytes.

opendatahub

maas-controller-79c79ff55f-qc5qq_8e6be790-7759-422b-8547-989be9fe5941

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-79c79ff55f-qc5qq_8e6be790-7759-422b-8547-989be9fe5941 became leader

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

FailedMount

MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-4qznm

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

FailedMount

MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

replicaset-controller

maas-controller-79c79ff55f

SuccessfulDelete

Deleted pod: maas-controller-79c79ff55f-qc5qq

opendatahub

kubelet

maas-controller-79c79ff55f-qc5qq

Killing

Stopping container manager

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-79c79ff55f from 1 to 0

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.216s (13.216s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-4qznm

Created

Container created

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-gjl74

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gjl74

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gjl74

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-gjl74

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

opendatahub

maas-controller-8677bb4cb4-5d7s9_8df2d7a0-f9b0-4722-94f2-bc87d7f3adc1

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-8677bb4cb4-5d7s9_8df2d7a0-f9b0-4722-94f2-bc87d7f3adc1 became leader

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gjl74

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gjl74

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-gjl74

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-q6hlp

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-q6hlp

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-l5vpc

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-lgdkj

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-gzdl5

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

multus

model-serving-api-75bd89fdfc-lgdkj

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-tkbx8

opendatahub

multus

kserve-controller-manager-776cc66d47-gzdl5

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-lgdkj

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-tkbx8

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

kserve-controller-manager-776cc66d47-gzdl5

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

kubelet

odh-model-controller-769875db56-l5vpc

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

multus

odh-model-controller-769875db56-l5vpc

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-tkbx8

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

opendatahub

kubelet

kserve-controller-manager-776cc66d47-gzdl5

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.744s (2.744s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-lgdkj

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-lgdkj

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 3.263s (3.263s including waiting). Image size: 184925761 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-gzdl5

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-lgdkj

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-gzdl5

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-l5vpc

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-l5vpc

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 4.132s (4.132s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-l5vpc

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-tkbx8

Started

Container started

opendatahub

kserve-controller-manager-776cc66d47-gzdl5_09ed0799-fbd4-416e-a503-3f97f101e758

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-gzdl5_09ed0799-fbd4-416e-a503-3f97f101e758 became leader

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-tkbx8

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-tkbx8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 5.224s (5.224s including waiting). Image size: 235100981 bytes.

opendatahub

odh-model-controller-769875db56-l5vpc_190bd209-7a4d-4ed8-bc90-95dd4385ea85

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-l5vpc_190bd209-7a4d-4ed8-bc90-95dd4385ea85 became leader

opendatahub

llmisvc-controller-manager-869d5dcc5d-tkbx8_da465967-6be4-46b3-8f3e-d7483f64943a

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-tkbx8_da465967-6be4-46b3-8f3e-d7483f64943a became leader

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

ProbeError

Readiness probe error: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Unhealthy

Readiness probe failed: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Started

Container started
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-659cdbc977 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-9bdb54dbc from 0 to 1

kuadrant-system

replicaset-controller

authorino-9bdb54dbc

SuccessfulCreate

Created pod: authorino-9bdb54dbc-72v2t

kuadrant-system

replicaset-controller

authorino-659cdbc977

SuccessfulDelete

Deleted pod: authorino-659cdbc977-49rwz

kuadrant-system

replicaset-controller

authorino-659cdbc977

SuccessfulCreate

Created pod: authorino-659cdbc977-49rwz

kuadrant-system

multus

authorino-6f6944bcdd-lwm7l

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-659cdbc977 from 1 to 0

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-lwm7l

kuadrant-system

kubelet

authorino-6f6944bcdd-lwm7l

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-lwm7l

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

multus

authorino-659cdbc977-49rwz

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-659cdbc977-49rwz

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-9bdb54dbc-72v2t

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-9bdb54dbc-72v2t

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-9bdb54dbc-72v2t

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.22s (1.22s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6f6944bcdd-lwm7l

Started

Container started

kuadrant-system

kubelet

authorino-659cdbc977-49rwz

Created

Container created

kuadrant-system

kubelet

authorino-659cdbc977-49rwz

Started

Container started

kuadrant-system

kubelet

authorino-659cdbc977-49rwz

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.426s (1.426s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6f6944bcdd-lwm7l

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.696s (1.696s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6f6944bcdd-lwm7l

Created

Container created

kuadrant-system

kubelet

authorino-9bdb54dbc-72v2t

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-lwm7l

Killing

Stopping container authorino
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

ProbeError

Startup probe error: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Unhealthy

Startup probe failed: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused

kuadrant-system

kubelet

authorino-659cdbc977-49rwz

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-56c6c9ddd5-bbc54

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-bbc54

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

kubelet

authorino-9bdb54dbc-72v2t

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-gstds

opendatahub

multus

llmisvc-controller-manager-6466c554d8-gstds

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gstds

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gstds

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gstds

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 738ms (738ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-gstds

Created

Container created

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-tkbx8

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-tkbx8

Killing

Stopping container manager

opendatahub

multus

ai-gateway-controller-5d8dddb676-qkgcd

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1
(x2)

opendatahub

kubelet

maas-controller-8677bb4cb4-5d7s9

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:36c8291eb6277ee16bcd2bc119baac75c285cfe906a702d29217a2100731824b"

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-qkgcd

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-qkgcd

opendatahub

kubelet

maas-controller-8677bb4cb4-5d7s9

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:36c8291eb6277ee16bcd2bc119baac75c285cfe906a702d29217a2100731824b" in 971ms (971ms including waiting). Image size: 200141006 bytes.
(x2)

opendatahub

kubelet

maas-controller-8677bb4cb4-5d7s9

Created

Container created
(x2)

opendatahub

kubelet

maas-controller-8677bb4cb4-5d7s9

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-qkgcd

Created

Container created

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-qkgcd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 3.295s (3.295s including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-qkgcd

Started

Container started

opendatahub

ai-gateway-controller-5d8dddb676-qkgcd_f7db853f-c445-454c-bbb6-13196ea147d3

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-qkgcd_f7db853f-c445-454c-bbb6-13196ea147d3 became leader

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-7bb9489676 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

SuccessfulCreate

Created pod: payload-processing-7bb9489676-tv42d
(x7)

openshift-ingress

replicaset-controller

payload-processing-7bb9489676

FailedCreate

Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-8ndrn
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

multus

payload-pre-processing-7b98bb678c-8ndrn

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-8ndrn

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-7bb9489676-tv42d

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-7bb9489676-tv42d

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-8ndrn

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.73s (2.73s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-8ndrn

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-8ndrn

Started

Container started

openshift-ingress

kubelet

payload-processing-7bb9489676-tv42d

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.741s (2.741s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-7bb9489676-tv42d

Started

Container started

openshift-ingress

kubelet

payload-processing-7bb9489676-tv42d

Created

Container created

opendatahub

llmisvc-controller-manager-6466c554d8-gstds_e147355f-1e7d-4d10-87e9-c375b7716c31

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-gstds_e147355f-1e7d-4d10-87e9-c375b7716c31 became leader

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-6d4cf6d5f from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-6d4cf6d5f

SuccessfulCreate

Created pod: ai-gateway-controller-6d4cf6d5f-9wgr6

opendatahub

multus

ai-gateway-controller-6d4cf6d5f-9wgr6

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-6d4cf6d5f-9wgr6

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-6d4cf6d5f-9wgr6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 955ms (955ms including waiting). Image size: 184136103 bytes.

opendatahub

kubelet

ai-gateway-controller-6d4cf6d5f-9wgr6

Started

Container started

opendatahub

kubelet

ai-gateway-controller-6d4cf6d5f-9wgr6

Created

Container created

opendatahub

maas-controller-8677bb4cb4-5d7s9_f36f3d66-32ab-4155-b195-db13c05509ff

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-8677bb4cb4-5d7s9_f36f3d66-32ab-4155-b195-db13c05509ff became leader

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-qkgcd

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-qkgcd

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.21s (2.21s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-b92k6

Created

Container created

opendatahub

ai-gateway-controller-6d4cf6d5f-9wgr6_a84e8a94-45c0-41a8-93c7-a022a422a5e0

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-6d4cf6d5f-9wgr6_a84e8a94-45c0-41a8-93c7-a022a422a5e0 became leader

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-hvhhm

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-hvhhm

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-hvhhm

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-hvhhm

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.596s (1.596s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-hvhhm

Started

Container started

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-hvhhm

Created

Container created

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-xgzhk

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-xgzhk

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xgzhk

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xgzhk

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 710ms (710ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xgzhk

Created

Container created

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-xgzhk

Started

Container started

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-z22gn

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-z22gn

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-z22gn

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-z22gn

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-z22gn

Started

Container started

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-z22gn

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 574ms (574ms including waiting). Image size: 98346788 bytes.

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 584ms (584ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8cmsql

Started

Container started

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-kn75t

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

multus

echo-simulator-kserve-64b6f8f96c-kn75t

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

llm

kubelet

echo-simulator-kserve-64b6f8f96c-kn75t

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

kubelet

echo-simulator-kserve-64b6f8f96c-kn75t

Created

Container created

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

kubelet

echo-simulator-kserve-64b6f8f96c-kn75t

Started

Container started

llm

kubelet

echo-simulator-kserve-64b6f8f96c-kn75t

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.67s (2.67s including waiting). Image size: 78208878 bytes.

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 622ms (622ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-5rb7k

Started

Container started

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 587ms (587ms including waiting). Image size: 98346788 bytes.

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-jsnqk

Started

Container started

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-xbf55

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-xbf55

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-hw8r9

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-778b7b5c47 from 1 to 4

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-kh4xr

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xbf55

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xbf55

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xbf55

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.07s (4.07s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

Started

Container started

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xbf55

ProbeError

Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xbf55

Unhealthy

Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

kuadrant-system

replicaset-controller

authorino-65457c7d56

SuccessfulCreate

Created pod: authorino-65457c7d56-kdp2h
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

Unhealthy

Startup probe failed: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused

kuadrant-system

kubelet

authorino-65457c7d56-kdp2h

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

ProbeError

Startup probe error: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused body:

kuadrant-system

multus

authorino-65457c7d56-kdp2h

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-65457c7d56-kdp2h

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.392s (1.392s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-9bdb54dbc-72v2t

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-65457c7d56-kdp2h

Created

Container created

kuadrant-system

replicaset-controller

authorino-9bdb54dbc

SuccessfulDelete

Deleted pod: authorino-9bdb54dbc-72v2t
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-9bdb54dbc from 1 to 0

kuadrant-system

kubelet

authorino-65457c7d56-kdp2h

Started

Container started

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-hs2x2

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-8wpzd

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-778b7b5c47 from 4 to 6

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

Unhealthy

Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

ProbeError

Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

Unhealthy

Startup probe failed: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

ProbeError

Startup probe error: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-f4d202

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-f4d202

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-fa0149

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-fa0149

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-82706f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-82706f

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w4-f4d202-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-f4d202-openshift-default-6f686cbcdf from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf

SuccessfulCreate

Created pod: e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

openshift-ingress

replicaset-controller

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85

SuccessfulCreate

Created pod: e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

openshift-ingress

deployment-controller

e2e-worker-w3-fa0149-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-fa0149-openshift-default-84d4d9bf85 from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 0 to 1

openshift-ingress

multus

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-7af38e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7af38e-openshift-default-765d77d66f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-7af38e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-7af38e

openshift-ingress

replicaset-controller

e2e-worker-w0-7af38e-openshift-default-765d77d66f

SuccessfulCreate

Created pod: e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

openshift-ingress

multus

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w1-795045

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-795045

openshift-ingress

multus

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

openshift-ingress

deployment-controller

e2e-worker-w1-795045-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-795045-openshift-default-85dbf685bc from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-f4d202

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-f4d202-78c5f7d5d from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-f4d202: serviceaccount "payload-processing-e2e-worker-w4-f4d202" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-f4d202: serviceaccount "payload-processing-e2e-worker-w4-f4d202" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-f4d202

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-795045-openshift-default-85dbf685bc

SuccessfulCreate

Created pod: e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.58s (2.58s including waiting). Image size: 125353963 bytes.

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-82706f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-82706f-76566d795f from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-82706f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-82706f-7b894c54f6 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-82706f-76566d795f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-82706f-76566d795f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-82706f: serviceaccount "payload-processing-e2e-worker-w2-82706f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-82706f-76566d795f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-82706f-7b894c54f6

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-82706f-7b894c54f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-82706f: serviceaccount "payload-processing-e2e-worker-w2-82706f" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-82706f-7b894c54f6

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

openshift-ingress

multus

payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-fa0149

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-fa0149-84f79cbb9d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw

Started

Container started
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-fa0149-768675769f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-fa0149-768675769f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-fa0149: serviceaccount "payload-processing-e2e-worker-w3-fa0149" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-fa0149: serviceaccount "payload-processing-e2e-worker-w3-fa0149" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-fa0149

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-fa0149-768675769f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-fa0149-768675769f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

openshift-ingress

multus

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-7af38e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-7af38e-848749bdcd

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-7af38e-848749bdcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-7af38e: serviceaccount "payload-processing-e2e-worker-w0-7af38e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-7af38e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-7af38e-848749bdcd from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-7af38e: serviceaccount "payload-processing-e2e-worker-w0-7af38e" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-7af38e-848749bdcd

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w1-795045

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w1-795045-5959688df9 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w1-795045

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w1-795045-7486fbc648 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-795045-7486fbc648

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w1-795045-7486fbc648-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-795045: serviceaccount "payload-processing-e2e-worker-w1-795045" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-795045-5959688df9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

openshift-ingress

multus

payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-795045-7486fbc648

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-795045-5959688df9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w1-795045-5959688df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-795045: serviceaccount "payload-processing-e2e-worker-w1-795045" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

ProbeError

Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

Unhealthy

Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

Unhealthy

Startup probe failed: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

ProbeError

Startup probe error: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-f4d202-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-f4d202-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

Unhealthy

Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

ProbeError

Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

ProbeError

Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

Unhealthy

Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-fa0149-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-fa0149-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7af38e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7af38e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-fa0149-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-fa0149-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-795045-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-795045-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-f4d202-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-f4d202-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-worker-w4-f4d202-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-f4d202-openshift-default-6f686cbcdf from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-f4d202-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf

SuccessfulCreate

Created pod: e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

Unhealthy

Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

ProbeError

Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-worker-w1-795045-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-795045-openshift-default-85dbf685bc from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-795045-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w1-795045-openshift-default-85dbf685bc

SuccessfulCreate

Created pod: e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

openshift-ingress

multus

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

Started

Container started

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

Unhealthy

Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

ProbeError

Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused body:

opendatahub

kubelet

test-curl-7117-3e8e8b

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7117-3e8e8b

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7117-3e8e8b

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.676s (2.676s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7117-3e8e8b

Created

Container created

opendatahub

kubelet

test-curl-7117-3e8e8b

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85

SuccessfulCreate

Created pod: e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

Created

Container created

openshift-ingress

multus

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

opendatahub

multus

test-curl-7117-92f403

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7117-92f403

Started

Container started

opendatahub

kubelet

test-curl-7117-92f403

Created

Container created

opendatahub

kubelet

test-curl-7117-92f403

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7117-3e8e8b

Killing

Stopping container test-curl-7117-3e8e8b

opendatahub

kubelet

test-curl-7117-92f403

Killing

Stopping container test-curl-7117-92f403
(x5)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

Unhealthy

Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

ProbeError

Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused body:

openshift-ingress

multus

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-a-c62113-openshift-default-76976bdfd9

SuccessfulCreate

Created pod: e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

openshift-ingress

deployment-controller

e2e-extmod-a-c62113-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-c62113-openshift-default-76976bdfd9 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-c62113

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-c62113

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xbf55

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hs2x2

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-778b7b5c47 from 6 to 4

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-xbf55

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-hs2x2

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-c62113

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-c62113-66b4575567 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-c62113

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-c62113-595cbf46fb from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-c62113-595cbf46fb

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-c62113-595cbf46fb

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-c62113-595cbf46fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-c62113: serviceaccount "payload-processing-e2e-extmod-a-c62113" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-c62113-66b4575567

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-c62113-66b4575567-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-c62113: serviceaccount "payload-processing-e2e-extmod-a-c62113" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-c62113-66b4575567

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

Unhealthy

Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

ProbeError

Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c62113-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c62113-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-extmod-a-06099f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-06099f-openshift-default-7cc677784f from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-06099f-openshift-default-7cc677784f

SuccessfulCreate

Created pod: e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-06099f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-06099f

openshift-ingress

multus

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c

SuccessfulCreate

Created pod: e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

openshift-ingress

deployment-controller

e2e-extmod-b-c2cde5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-c2cde5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-c2cde5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-06099f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-06099f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-06099f-696b476f84 from 0 to 1

openshift-ingress

multus

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-06099f-696b476f84

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-06099f-696b476f84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-06099f: serviceaccount "payload-processing-e2e-extmod-a-06099f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-06099f: serviceaccount "payload-processing-e2e-extmod-a-06099f" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-06099f-696b476f84

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-8wpzd

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-8wpzd

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-778b7b5c47 from 4 to 3

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-c2cde5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-c2cde5-56d5855b9d from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-c2cde5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-c2cde5: serviceaccount "payload-processing-e2e-extmod-b-c2cde5" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-c2cde5: serviceaccount "payload-processing-e2e-extmod-b-c2cde5" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf

Created

Container created
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c62113-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c62113-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

Started

Container started

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

Created

Container created

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 2 to 4

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

ProbeError

Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

Unhealthy

Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-778b7b5c47 from 3 to 2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-hw8r9

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-hw8r9

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

multus

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-f4d202-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-f4d202-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-f4d202-openshift-default-6f686cbcdf from 2 to 4

openshift-ingress

multus

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf

SuccessfulCreate

Created pod: e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

openshift-ingress

replicaset-controller

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf

SuccessfulCreate

Created pod: e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

Unhealthy

Readiness probe failed: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

ProbeError

Readiness probe error: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-586b974f6 from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

Started

Container started

models-as-a-service

replicaset-controller

payload-processing-external-model-586b974f6

SuccessfulCreate

Created pod: payload-processing-external-model-586b974f6-dt8ql
(x6)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

Unhealthy

Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

ProbeError

Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

Created

Container created
(x7)

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused
(x8)

models-as-a-service

replicaset-controller

payload-processing-external-model-586b974f6

FailedCreate

Error creating: pods "payload-processing-external-model-586b974f6-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

multus

payload-processing-external-model-586b974f6-dt8ql

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-586b974f6-dt8ql

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-586b974f6-dt8ql

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-586b974f6-dt8ql

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-5497dccdcc

SuccessfulCreate

Created pod: payload-processing-external-model-5497dccdcc-bbs4k

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5497dccdcc from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-5497dccdcc-bbs4k

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-5497dccdcc-bbs4k

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5497dccdcc-bbs4k

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-5497dccdcc-bbs4k

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

Unhealthy

Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

ProbeError

Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7af38e-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-7af38e-openshift-default-765d77d66f

SuccessfulCreate

Created pod: e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

openshift-ingress

deployment-controller

e2e-worker-w0-7af38e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7af38e-openshift-default-765d77d66f from 1 to 2
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-06099f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-06099f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

Started

Container started

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-586b974f6 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-586b974f6

SuccessfulDelete

Deleted pod: payload-processing-external-model-586b974f6-dt8ql

models-as-a-service

kubelet

payload-processing-external-model-586b974f6-dt8ql

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

Unhealthy

Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

ProbeError

Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

ProbeError

Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

Unhealthy

Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-06099f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-06099f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-c0ab5f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-c0ab5f

openshift-ingress

deployment-controller

e2e-extmod-b-c0ab5f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-c0ab5f-openshift-default-8977b7949 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-c0ab5f-openshift-default-8977b7949

SuccessfulCreate

Created pod: e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

openshift-ingress

multus

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c2cde5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c2cde5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-778b7b5c47 from 2 to 4

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-nbx74

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-nbx74
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-p7nkd

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-p7nkd

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nbx74

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-p7nkd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nbx74

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nbx74

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-p7nkd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-p7nkd

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 4 to 7

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

Created

Container created
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f: serviceaccount "payload-processing-e2e-extmod-b-c0ab5f" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-c0ab5f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-c0ab5f: serviceaccount "payload-processing-e2e-extmod-b-c0ab5f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-c0ab5f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nbx74

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nbx74

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-p7nkd

Unhealthy

Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-p7nkd

ProbeError

Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

ProbeError

Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

Unhealthy

Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c2cde5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c2cde5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-778b7b5c47-xml2m

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-778b7b5c47 from 4 to 8

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qpptj

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-qpptj

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qpptj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nwszr

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nwszr

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nwszr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-nwszr

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qpptj

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-xml2m

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xml2m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xml2m

Created

Container created

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-8ffbc55d8 from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xml2m

Started

Container started

models-as-a-service

replicaset-controller

payload-processing-external-model-8ffbc55d8

SuccessfulCreate

Created pod: payload-processing-external-model-8ffbc55d8-rqd7j

models-as-a-service

kubelet

payload-processing-external-model-8ffbc55d8-rqd7j

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-8ffbc55d8-rqd7j

Started

Container started

models-as-a-service

multus

payload-processing-external-model-8ffbc55d8-rqd7j

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-8ffbc55d8-rqd7j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-extmod-a-79badd-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-79badd-openshift-default-68df55b6df from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-79badd-openshift-default-68df55b6df

SuccessfulCreate

Created pod: e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-79badd

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-79badd

openshift-ingress

multus

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xml2m

Unhealthy

Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xml2m

ProbeError

Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

ProbeError

Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qpptj

ProbeError

Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

Unhealthy

Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

Unhealthy

Readiness probe failed: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qpptj

Unhealthy

Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-vkm88

ProbeError

Readiness probe error: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nwszr

Unhealthy

Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nwszr

ProbeError

Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-79badd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-79badd-5d88468c95 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-79badd-5f76959dc4

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-79badd-5f76959dc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-79badd: serviceaccount "payload-processing-e2e-extmod-a-79badd" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-79badd: serviceaccount "payload-processing-e2e-extmod-a-79badd" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-79badd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-79badd-5f76959dc4 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl

openshift-ingress

multus

payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z

AddedInterface

Add eth0 [10.132.0.84/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-79badd-5f76959dc4

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl

AddedInterface

Add eth0 [10.132.0.83/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 9; reason:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-778b7b5c47 from 8 to 9

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kdtlh

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-kdtlh

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kdtlh

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kdtlh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kdtlh

Unhealthy

Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kdtlh

ProbeError

Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-f4d202-6c57ff646c-4657x

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-f4d202-78c5f7d5d-vdvr6

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-xxh8g

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-4frdh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-dwdf2

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-f4d202-openshift-default-6f686cbcdf-gs7w4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

Unhealthy

Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

ProbeError

Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

ProbeError

Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

Unhealthy

Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c0ab5f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c0ab5f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

ProbeError

Readiness probe error: Get "http://10.133.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-kh4xr

Unhealthy

Readiness probe failed: Get "http://10.133.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

replicaset-controller

e2e-worker-w1-795045-openshift-default-85dbf685bc

SuccessfulDelete

Deleted pod: e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-795045-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w1-795045-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w1-795045-openshift-default-85dbf685bc from 2 to 1

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-99mrz

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

ProbeError

Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

Unhealthy

Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c0ab5f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-c0ab5f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c62113-595cbf46fb-6j4zt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c62113-66b4575567-mnkbz

Killing

Stopping container payload-pre-processing
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-79badd-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-79badd-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-extmod-b-27bb49-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-27bb49-openshift-default-c68db7497 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-27bb49

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-27bb49

openshift-ingress

replicaset-controller

e2e-extmod-b-27bb49-openshift-default-c68db7497

SuccessfulCreate

Created pod: e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

openshift-ingress

multus

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-a-c62113-openshift-default-76976bdfd9-vs5jm

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-27bb49

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-27bb49-588f49664b from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-27bb49-588f49664b

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-27bb49-588f49664b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-27bb49: serviceaccount "payload-processing-e2e-extmod-b-27bb49" not found
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-27bb49: serviceaccount "payload-processing-e2e-extmod-b-27bb49" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-27bb49

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz

AddedInterface

Add eth0 [10.132.0.87/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-27bb49-588f49664b

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b

AddedInterface

Add eth0 [10.132.0.86/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c2cde5-56d5855b9d-gcjmf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c2cde5-7f7fbf4dd7-dftdc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-b-c2cde5-openshift-default-76cd4cdf5c-hdl6w

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-795045-7486fbc648-z42w6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-795045-5959688df9-jvzxl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

ProbeError

Readiness probe error: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

Unhealthy

Readiness probe failed: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

Started

Container started
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-06099f-79f744f4c5-t8kjj

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-06099f-696b476f84-2nscz

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x5)

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

ProbeError

Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

Unhealthy

Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w1-795045-openshift-default-85dbf685bc-28qrm

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w3-fa0149-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-fa0149-openshift-default-84d4d9bf85 from 2 to 1

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85

SuccessfulDelete

Deleted pod: e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-fa0149-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-a-06099f-openshift-default-7cc677784f-s9wq6

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-27bb49-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-27bb49-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-hsln4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-c0ab5f-84b7456d4d-rgw8l

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-27bb49-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-27bb49-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-c0ab5f-57599c5f67-6pzd8

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-c0ab5f-openshift-default-8977b7949-b2qvt

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-fa0149-84f79cbb9d-zbn5v

Killing

Stopping container payload-processing
(x2)

openshift-ingress

deployment-controller

e2e-worker-w3-fa0149-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-fa0149-openshift-default-84d4d9bf85 from 1 to 2

openshift-ingress

multus

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

AddedInterface

Add eth0 [10.134.0.87/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85

SuccessfulCreate

Created pod: e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-fa0149-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

Created

Container created
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-fa0149-768675769f-hhxm4

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

ProbeError

Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

Unhealthy

Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-79badd-5d88468c95-5tkhl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-79badd-5f76959dc4-jhx4z

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-cnkt5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-fa0149-openshift-default-84d4d9bf85-2mpqn

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulDelete

Deleted pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-qzmz9

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulDelete

Deleted pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 7 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7af38e-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w0-7af38e-openshift-default-765d77d66f

SuccessfulDelete

Deleted pod: e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

openshift-ingress

deployment-controller

e2e-worker-w0-7af38e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-7af38e-openshift-default-765d77d66f from 2 to 1

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-lv8wh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-l8t67

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-79badd-openshift-default-68df55b6df-hh2fc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-27bb49-588f49664b-nq97b

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-27bb49-cf7dc9c59-bl4zz

Killing

Stopping container payload-pre-processing
(x33)

models-as-a-service

kubelet

payload-processing-external-model-8ffbc55d8-rqd7j

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulDelete

Deleted pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-4h4mn

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 5 to 4

openshift-ingress

kubelet

e2e-extmod-b-27bb49-openshift-default-c68db7497-9rfc6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 4 to 8

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

AddedInterface

Add eth0 [10.134.0.90/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Started

Container started

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

AddedInterface

Add eth0 [10.134.0.91/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Created

Container created
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-82706f-7b894c54f6-9dq42

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-82706f-76566d795f-jm7tw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

ProbeError

Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

ProbeError

Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Unhealthy

Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Unhealthy

Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

ProbeError

Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Unhealthy

Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Unhealthy

Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

ProbeError

Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

ProbeError

Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

ProbeError

Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-82706f-openshift-default

SuccessfulRescale

New size: 10; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 8 to 10

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

AddedInterface

Add eth0 [10.132.0.90/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

AddedInterface

Add eth0 [10.134.0.92/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

Unhealthy

Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

ProbeError

Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

Unhealthy

Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

ProbeError

Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-8df4k

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w2-82706f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-82706f-openshift-default-c4c5d579 from 0 to 1
(x3)

openshift-ingress

replicaset-controller

e2e-worker-w2-82706f-openshift-default-c4c5d579

FailedCreate

Error creating: pods "e2e-worker-w2-82706f-openshift-default-c4c5d579-" is forbidden: error looking up service account openshift-ingress/e2e-worker-w2-82706f-openshift-default: serviceaccount "e2e-worker-w2-82706f-openshift-default" not found

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w2-82706f-openshift-default-dockercfg-v862d); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jc9mq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w2-82706f-openshift-default-dockercfg-v862d); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-kxz2k

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-jhl59

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-j2g4p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-78qdf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-cgjz5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-q78dw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-zzjb8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-82706f-openshift-default-c4c5d579-v8hqs

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-nbx74

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-jjhm5

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-nbx74

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-xml2m

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-xml2m

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-jjhm5

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-778b7b5c47-m9d6q

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m9d6q

Killing

Stopping container istio-proxy
(x11)

models-as-a-service

kubelet

payload-processing-external-model-5497dccdcc-bbs4k

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-84cffd7d1248" not found

models-as-a-service

kubelet

payload-processing-external-model-8ffbc55d8-rqd7j

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-5497dccdcc-bbs4k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

Killing

Stopping container payload-pre-processing
(x9)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-778b7b5c47

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-778b7b5c47-m59rh
(x3)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-778b7b5c47 from 6 to 10
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m59rh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

AddedInterface

Add eth0 [10.134.0.99/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-m59rh

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m59rh

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

AddedInterface

Add eth0 [10.134.0.96/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m59rh

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

AddedInterface

Add eth0 [10.134.0.97/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

Started

Container started

openshift-ingress

multus

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-worker-w0-7af38e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-7af38e-openshift-default-765d77d66f from 1 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-7af38e-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w0-7af38e-openshift-default-765d77d66f

SuccessfulCreate

Created pod: e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

openshift-ingress

replicaset-controller

e2e-worker-w0-7af38e-openshift-default-765d77d66f

SuccessfulCreate

Created pod: e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

openshift-ingress

multus

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

Started

Container started
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m59rh

ProbeError

Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

Unhealthy

Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

ProbeError

Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

Unhealthy

Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

ProbeError

Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

ProbeError

Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

Unhealthy

Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m59rh

Unhealthy

Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

ProbeError

Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

Unhealthy

Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-l8hwj

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

Unhealthy

Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused
(x3)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

ProbeError

Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m59rh

Unhealthy

Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

Unhealthy

Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-pnm2w

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-m59rh

ProbeError

Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-7af38e-848749bdcd-2h9nh

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-qnxqv

ProbeError

Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

Unhealthy

Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-5x8z8

ProbeError

Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-7af38e-openshift-default-765d77d66f-djfrh

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-7af38e-fdb56c76c-lgzvt

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

Unhealthy

Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-778b7b5c47-s8k5g

ProbeError

Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body: