Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95 to ip-10-0-134-9.ec2.internal

models-as-a-service

payload-processing-external-model-5bcd7997-n5n4n

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5bcd7997-n5n4n to ip-10-0-134-99.ec2.internal

ai-tenants

e2e-autores-147981

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-autores-147981: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-explicit-5a4525

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-explicit-5a4525: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-944506

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-944506: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-9ba664

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-9ba664: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-bf0b0d

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-bf0b0d: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-12b8da

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-12b8da: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-2bb4ef

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-2bb4ef: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-673ba8

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-673ba8: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-a-758617

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-a-758617: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-b-854b9e

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-b-854b9e: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-a-b21aae

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-a-b21aae: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-b-be6018

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-b-be6018: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-778b2bce

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-778b2bce: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-90a96161

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-90a96161: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-a-de8cc7

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-a-de8cc7: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-b-514524

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-b-514524: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w3-e07583

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w3-e07583: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w4-f1e1c2

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w4-f1e1c2: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w3-5ba53f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w3-5ba53f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w4-623b4b

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w4-623b4b: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-555d1e

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-555d1e: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w1-f72bc6

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w1-f72bc6: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-4be645

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-4be645: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-866159

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-866159: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-c0c28f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-c0c28f: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-74djh

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-74djh to ip-10-0-134-9.ec2.internal

opendatahub

maas-controller-5db8dc9fd8-jdk62

Scheduled

Successfully assigned opendatahub/maas-controller-5db8dc9fd8-jdk62 to ip-10-0-129-115.ec2.internal

opendatahub

maas-controller-5db8dc9fd8-zlxn4

Scheduled

Successfully assigned opendatahub/maas-controller-5db8dc9fd8-zlxn4 to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722 to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w to ip-10-0-134-99.ec2.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-lvdx4

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-lvdx4 to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-bbc4b5f5d-l54ph

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-bbc4b5f5d-l54ph to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-7b98bb678c-2btkh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-2btkh to ip-10-0-134-99.ec2.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-mj5lp

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-mj5lp to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-574bdd7c8c-s7zkh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-574bdd7c8c-s7zkh to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8 to ip-10-0-134-9.ec2.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb to ip-10-0-129-115.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-autores-147981-openshift-default-76d9954467-94hjz

Scheduled

Successfully assigned openshift-ingress/e2e-autores-147981-openshift-default-76d9954467-94hjz to ip-10-0-134-9.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-zvq6k

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-zvq6k to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-zdbqj

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-zdbqj to ip-10-0-129-115.ec2.internal

opendatahub

maas-controller-5db8dc9fd8-76ms5

Scheduled

Successfully assigned opendatahub/maas-controller-5db8dc9fd8-76ms5 to ip-10-0-134-9.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-t9gjp

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-t9gjp to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf to ip-10-0-134-99.ec2.internal

kuadrant-system

authorino-56c6c9ddd5-7dxb9

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-7dxb9 to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-autores-147981-77f5d75b89-sg26f

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-147981-77f5d75b89-sg26f to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx to ip-10-0-134-99.ec2.internal

opendatahub

maas-controller-64d68d6d56-76wsv

Scheduled

Successfully assigned opendatahub/maas-controller-64d68d6d56-76wsv to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77 to ip-10-0-134-9.ec2.internal

kuadrant-system

authorino-57f7f8659f-sn9rq

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-sn9rq to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng to ip-10-0-134-99.ec2.internal

opendatahub

model-serving-api-75bd89fdfc-4kw4d

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-4kw4d to ip-10-0-134-9.ec2.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-vqsrv

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-vqsrv to ip-10-0-134-9.ec2.internal

opendatahub

odh-model-controller-769875db56-z2nb2

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-z2nb2 to ip-10-0-134-9.ec2.internal

opendatahub

test-curl-7807-0ab152

Scheduled

Successfully assigned opendatahub/test-curl-7807-0ab152 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-8f7f4697c-89t4w

Scheduled

Successfully assigned openshift-ingress/payload-processing-8f7f4697c-89t4w to ip-10-0-134-99.ec2.internal

kuadrant-system

authorino-64c547bcd4-s7w4p

Scheduled

Successfully assigned kuadrant-system/authorino-64c547bcd4-s7w4p to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4 to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm to ip-10-0-129-115.ec2.internal

kuadrant-system

authorino-6889cbd8c4-7696f

Scheduled

Successfully assigned kuadrant-system/authorino-6889cbd8c4-7696f to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84 to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d to ip-10-0-134-9.ec2.internal

kuadrant-system

authorino-6f6944bcdd-hv5z8

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-hv5z8 to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Scheduled

Successfully assigned openshift-ingress/e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8 to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4 to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46 to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m to ip-10-0-134-99.ec2.internal

kuadrant-system

authorino-c75459655-fdx74

Scheduled

Successfully assigned kuadrant-system/authorino-c75459655-fdx74 to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5 to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5 to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8 to ip-10-0-134-99.ec2.internal

ai-tenants

e2e-api-a-d0c9b9

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-a-d0c9b9: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79 to ip-10-0-134-9.ec2.internal

kuadrant-system

authorino-operator-7b7bb4bc77-gbpwl

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-gbpwl to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-849d6b95cc-pc2n9

Scheduled

Successfully assigned openshift-ingress/payload-processing-849d6b95cc-pc2n9 to ip-10-0-134-99.ec2.internal

opendatahub

test-curl-7807-2c2ad3

Scheduled

Successfully assigned opendatahub/test-curl-7807-2c2ad3 to ip-10-0-134-99.ec2.internal

opendatahub

test-curl-7807-4143b6

Scheduled

Successfully assigned opendatahub/test-curl-7807-4143b6 to ip-10-0-134-9.ec2.internal

opendatahub

llmisvc-controller-manager-6466c554d8-fttqq

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-fttqq to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-processing-55fbbbf9dc-rx5hk

Scheduled

Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-rx5hk to ip-10-0-134-99.ec2.internal

opendatahub

test-curl-7807-e92a4a

Scheduled

Successfully assigned opendatahub/test-curl-7807-e92a4a to ip-10-0-134-99.ec2.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-2p5lx

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-2p5lx to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-q8r79

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-q8r79 to ip-10-0-134-9.ec2.internal

llm

echo-simulator-kserve-64b6f8f96c-92h45

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-92h45 to ip-10-0-134-99.ec2.internal

opendatahub

test-curl-7813-09c805

Scheduled

Successfully assigned opendatahub/test-curl-7813-09c805 to ip-10-0-134-99.ec2.internal

opendatahub

test-curl-7813-b1ec9c

Scheduled

Successfully assigned opendatahub/test-curl-7813-b1ec9c to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm to ip-10-0-129-115.ec2.internal

opendatahub

test-curl-7816-469647

Scheduled

Successfully assigned opendatahub/test-curl-7816-469647 to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx to ip-10-0-129-115.ec2.internal

opendatahub

test-curl-7816-4986cc

Scheduled

Successfully assigned opendatahub/test-curl-7816-4986cc to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8 to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7 to ip-10-0-129-115.ec2.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-jfj67

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-jfj67 to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

Scheduled

Successfully assigned openshift-ingress/e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9 to ip-10-0-134-9.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-pxflc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-pxflc to ip-10-0-134-9.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-jbt2l to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4 to ip-10-0-134-99.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-clxsz

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-clxsz to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-cknl5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-cknl5 to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-944506-openshift-default-855597c485-fdk9x to ip-10-0-134-9.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-bxtwk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-bxtwk to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-b8n6c

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-b8n6c to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-9pqlk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-9pqlk to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-97tgx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-97tgx to ip-10-0-134-9.ec2.internal

opendatahub

test-curl-7816-a964bc

Scheduled

Successfully assigned opendatahub/test-curl-7816-a964bc to ip-10-0-134-9.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-8q5hv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-8q5hv to ip-10-0-134-99.ec2.internal

opendatahub

kserve-controller-manager-776cc66d47-7xw9r

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-7xw9r to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445 to ip-10-0-129-115.ec2.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-mgjcj

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-mgjcj to ip-10-0-134-9.ec2.internal

opendatahub

test-curl-7816-f108c6

Scheduled

Successfully assigned opendatahub/test-curl-7816-f108c6 to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb to ip-10-0-134-99.ec2.internal

opendatahub

test-curl-7816-f861bc

Scheduled

Successfully assigned opendatahub/test-curl-7816-f861bc to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5 to ip-10-0-134-9.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-4flwn

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-4flwn to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5 to ip-10-0-134-99.ec2.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-c2cbh

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-c2cbh to ip-10-0-134-99.ec2.internal

opendatahub

ai-gateway-controller-6cc7b895d5-5ldn8

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-6cc7b895d5-5ldn8 to ip-10-0-134-99.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-fgnlf

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-fgnlf to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-89nq9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-89nq9 to ip-10-0-129-115.ec2.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw to ip-10-0-129-115.ec2.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-78cdb466dc-d84mp to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd to ip-10-0-134-99.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-4q677

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-4q677 to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-4nbcz

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-4nbcz to ip-10-0-129-115.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-6jkfj

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-6jkfj to ip-10-0-129-115.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-2m48q

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-2m48q to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526 to ip-10-0-134-9.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55fbd65575-259d8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-259d8 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j to ip-10-0-134-99.ec2.internal

opendatahub

ai-gateway-controller-5d8dddb676-xjj8d

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-xjj8d to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

Scheduled

Successfully assigned openshift-ingress/e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd to ip-10-0-134-9.ec2.internal

kuadrant-system

limitador-limitador-74dfcfb894-9zhx5

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-9zhx5 to ip-10-0-134-9.ec2.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-nnfxn

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-nnfxn to ip-10-0-134-99.ec2.internal

llm

e2e-same-id-b-2e2eed0e

ModelNameConflictResolved

Model identity is no longer shared with any other MaaSModelRef in this namespace

opendatahub-operator-system

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-78cdb466dc-dnwzd to ip-10-0-134-99.ec2.internal

llm

e2e-same-id-b-2e2eed0e

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-a-2e2eed0e

openshift-ingress

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h to ip-10-0-129-115.ec2.internal

openshift-jobset-operator

jobset-controller-manager-67b8558cc4-5qlrn

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-67b8558cc4-5qlrn to ip-10-0-134-99.ec2.internal

kuadrant-system

limitador-limitador-7c9b87d574-ghltt

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-ghltt to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6 to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7 to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Scheduled

Successfully assigned openshift-ingress/e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97 to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts to ip-10-0-134-99.ec2.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-78cdb466dc-fxs5r to ip-10-0-129-115.ec2.internal

llm

e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq

Scheduled

Successfully assigned llm/e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq to ip-10-0-134-99.ec2.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-9dxhn

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-9dxhn to ip-10-0-129-115.ec2.internal

llm

e2e-same-id-a-2e2eed0e

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-b-2e2eed0e

openshift-ingress

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95 to ip-10-0-134-99.ec2.internal

models-as-a-service

payload-processing-external-model-85d6f9ccbb-pssz5

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-85d6f9ccbb-pssz5 to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Scheduled

Successfully assigned openshift-ingress/e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc to ip-10-0-129-115.ec2.internal

models-as-a-service

payload-processing-external-model-54cc564d84-2md8z

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-54cc564d84-2md8z to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg to ip-10-0-134-99.ec2.internal

models-as-a-service

payload-processing-external-model-85d6f9ccbb-86w9p

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-85d6f9ccbb-86w9p to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s to ip-10-0-134-99.ec2.internal

openshift-ingress

router-default-7899fcdf65-hl87t

Scheduled

Successfully assigned openshift-ingress/router-default-7899fcdf65-hl87t to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq to ip-10-0-134-99.ec2.internal

openshift-ingress

router-default-7899fcdf65-hl87t

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-7899fcdf65-hl87t

FailedScheduling

no nodes available to schedule pods

openshift-ingress

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-mt-90a96161-694f45b47c-t985m

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-90a96161-694f45b47c-t985m to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt to ip-10-0-129-115.ec2.internal

models-as-a-service

payload-processing-external-model-8569d75f4b-b6ftn

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-8569d75f4b-b6ftn to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw to ip-10-0-134-99.ec2.internal

ai-tenants

e2e-api-b-928a7a

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-b-928a7a: MaasTenantConfig is not available to restore revocation operands

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82 to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2 to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl

Scheduled

Successfully assigned openshift-ingress/e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq to ip-10-0-134-9.ec2.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-kwrdx

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-kwrdx to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

Scheduled

Successfully assigned openshift-ingress/e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

Scheduled

Successfully assigned openshift-ingress/e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8 to ip-10-0-134-9.ec2.internal

openshift-ingress

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-758617-openshift-default-754657d5bc-vcf88 to ip-10-0-129-115.ec2.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-t8szh

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-t8szh to ip-10-0-134-99.ec2.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-fqd5g

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-fqd5g to ip-10-0-134-9.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb to ip-10-0-129-115.ec2.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-4zm5n

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-4zm5n to ip-10-0-129-115.ec2.internal

llm

e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z

Scheduled

Successfully assigned llm/e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk to ip-10-0-129-115.ec2.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-9lb82

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9lb82 to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn to ip-10-0-134-9.ec2.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rdlmf to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

Scheduled

Successfully assigned openshift-ingress/e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj to ip-10-0-134-99.ec2.internal

openshift-ingress

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq to ip-10-0-134-99.ec2.internal

openshift-ingress

kube-auth-proxy-8f9c48f5f-jlflv

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-8f9c48f5f-jlflv to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5 to ip-10-0-129-115.ec2.internal

models-as-a-service

payload-processing-external-model-5c4dd54948-rb9gr

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5c4dd54948-rb9gr to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9 to ip-10-0-129-115.ec2.internal

models-as-a-service

payload-processing-external-model-5c4dd54948-xjxbh

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5c4dd54948-xjxbh to ip-10-0-129-115.ec2.internal

openshift-ingress

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4 to ip-10-0-129-115.ec2.internal

models-as-a-service

payload-processing-external-model-7c999f66-ks4pr

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7c999f66-ks4pr to ip-10-0-129-115.ec2.internal

openshift-ingress

kube-auth-proxy-8f9c48f5f-prw2d

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-8f9c48f5f-prw2d to ip-10-0-134-9.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-bb66448f7-4p8z8

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-bb66448f7-4p8z8 to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx to ip-10-0-129-115.ec2.internal

llm

e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb

Scheduled

Successfully assigned llm/e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6 to ip-10-0-134-99.ec2.internal

models-as-a-service

payload-processing-external-model-747f596d58-lncqr

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-747f596d58-lncqr to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784 to ip-10-0-129-115.ec2.internal

llm

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

Scheduled

Successfully assigned llm/e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q to ip-10-0-129-115.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r to ip-10-0-134-9.ec2.internal

models-as-a-service

payload-processing-external-model-778c654c9-lrkxv

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-778c654c9-lrkxv to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2 to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z to ip-10-0-134-99.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g to ip-10-0-134-99.ec2.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv to ip-10-0-134-99.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-7899fcdf65 from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-7899fcdf65

SuccessfulCreate

Created pod: router-default-7899fcdf65-hl87t
(x2)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-7899fcdf65-hl87t

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-7899fcdf65-hl87t

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x2)

openshift-ingress

kubelet

router-default-7899fcdf65-hl87t

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-7899fcdf65-hl87t

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-7899fcdf65-hl87t

Started

Container started

openshift-ingress

kubelet

router-default-7899fcdf65-hl87t

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

multus

router-default-7899fcdf65-hl87t

AddedInterface

Add eth0 [10.132.0.13/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-7899fcdf65-hl87t

Created

Container created

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-t8szh

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-t8szh

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-t8szh

AddedInterface

Add eth0 [10.132.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-t8szh

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-t8szh

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.534s (2.534s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-t8szh

Started

Container started

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-645b5cfbff from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-clxsz

AddedInterface

Add eth0 [10.134.0.20/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-645b5cfbff

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-clxsz

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 0 to 1

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.531s (9.531s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.773s (9.773s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr

Created

Container created

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-645b5cfbff-gb6mr

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-78cdb466dc

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-78cdb466dc-fxs5r

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-78cdb466dc

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-78cdb466dc-dnwzd

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

AddedInterface

Add eth0 [10.132.0.26/23] from ovn-kubernetes

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-78cdb466dc from 0 to 3

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-78cdb466dc

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-78cdb466dc-d84mp

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-78cdb466dc-d84mp

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

NeedsReinstall

calculated deployment install is bad
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

operator-lifecycle-manager

install-vcv6w

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Created

Container created
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.762s (2.762s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.851s (2.851s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.824s (2.824s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 354ms (354ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-fxs5r

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 393ms (393ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 307ms (307ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-d84mp

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-78cdb466dc-dnwzd

Created

Container created

opendatahub-operator-system

opendatahub-operator-controller-manager-78cdb466dc-fxs5r_4c4189e9-cbf9-46da-959c-7aa9353e71e9

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-78cdb466dc-fxs5r_4c4189e9-cbf9-46da-959c-7aa9353e71e9 became leader

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-mj5lp
(x9)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-mj5lp

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-mj5lp

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-lvdx4

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-6jkfj

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-6jkfj

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-lvdx4

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-lvdx4

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-6jkfj

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-mj5lp

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 2.943s (2.943s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-mj5lp

Created

Container created

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-mj5lp

Started

Container started

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-lvdx4

Created

Container created

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-lvdx4

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-lvdx4

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.762s (3.762s including waiting). Image size: 309650901 bytes.

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-jfj67

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-jfj67

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-6jkfj_7ca0c5ee-a7c0-4cb9-aedd-cd09843b0af5

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-6jkfj_7ca0c5ee-a7c0-4cb9-aedd-cd09843b0af5 became leader
(x11)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-6jkfj

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-6jkfj

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-6jkfj

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 4.486s (4.486s including waiting). Image size: 424244516 bytes.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jfj67

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-fqd5g

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

kuadrant-system

dns-operator-controller-manager-8557cbdb84-jfj67_400a0a70-44d2-4429-a680-c11e84556ec3

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-jfj67_400a0a70-44d2-4429-a680-c11e84556ec3 became leader

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jfj67

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 1.549s (1.549s including waiting). Image size: 190804149 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jfj67

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-jfj67

Started

Container started

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-fqd5g

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-fqd5g

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-mgjcj

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-mgjcj

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-gbpwl

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-mgjcj

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

authorino-operator-7b7bb4bc77-gbpwl

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-fqd5g

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-gbpwl

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-fqd5g

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 2.795s (2.795s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-74djh

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-fqd5g

Started

Container started

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

cert-manager

multus

cert-manager-597b95cb84-74djh

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-gbpwl

Started

Container started

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-fqd5g_6a6bf5c6-7898-491a-8e12-84254c7da7c7 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-gbpwl

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-gbpwl

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 1.329s (1.329s including waiting). Image size: 167738465 bytes.

cert-manager

kubelet

cert-manager-597b95cb84-74djh

Started

Container started

cert-manager

kubelet

cert-manager-597b95cb84-74djh

Created

Container created

cert-manager

kubelet

cert-manager-597b95cb84-74djh

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

kuadrant-system

authorino-operator-7b7bb4bc77-gbpwl_dd7cd39f-4312-4a3e-a50f-f43558f7c40b

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-gbpwl_dd7cd39f-4312-4a3e-a50f-f43558f7c40b became leader

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-9dxhn

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-9dxhn

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-9dxhn

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-mgjcj

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-mgjcj

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 5.323s (5.323s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-mgjcj

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-9dxhn

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 2.067s (2.067s including waiting). Image size: 177967631 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-9dxhn

Started

Container started

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-9dxhn

Created

Container created

kuadrant-system

limitador-operator-controller-manager-5778bb4449-9dxhn_47b1b834-4d16-4893-8a94-14ee4b748ce0

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-9dxhn_47b1b834-4d16-4893-8a94-14ee4b748ce0 became leader

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-4flwn

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-4flwn

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-4flwn

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-4flwn

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-4flwn

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-prw2d

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-jlflv

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

replicaset-controller

kube-auth-proxy-8f9c48f5f

SuccessfulCreate

Created pod: kube-auth-proxy-8f9c48f5f-jlflv

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-bb66448f7 from 0 to 1

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-bb66448f7

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-bb66448f7-4p8z8

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-8f9c48f5f from 0 to 2

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-ingress

multus

kube-auth-proxy-8f9c48f5f-prw2d

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-ingress

multus

kube-auth-proxy-8f9c48f5f-jlflv

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

kube-auth-proxy-8f9c48f5f

SuccessfulCreate

Created pod: kube-auth-proxy-8f9c48f5f-prw2d

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-bb66448f7-4p8z8

Created

Container created

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-bb66448f7-4p8z8

Started

Container started

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-bb66448f7-4p8z8

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-bb66448f7-4p8z8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-prw2d

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-jlflv

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-jlflv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.312s (2.313s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-prw2d

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.405s (2.405s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-jlflv

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-8f9c48f5f-prw2d

Started

Container started

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-6jkfj

Killing

Stopping container manager

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-6jkfj

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-2p5lx

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-2p5lx

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-2p5lx

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-2p5lx

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-2p5lx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.297s (2.297s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-2p5lx

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-4flwn_c121843c-6829-423b-834f-70f4a75bf982

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-4flwn_c121843c-6829-423b-834f-70f4a75bf982 became leader

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-ds5jw"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-shrg4"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-9zhx5

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-67b8558cc4

SuccessfulCreate

Created pod: jobset-controller-manager-67b8558cc4-5qlrn

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9zhx5

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

multus

limitador-limitador-74dfcfb894-9zhx5

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-sn9rq

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

multus

jobset-controller-manager-67b8558cc4-5qlrn

AddedInterface

Add eth0 [10.132.0.28/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-67b8558cc4 from 0 to 1

openshift-jobset-operator

kubelet

jobset-controller-manager-67b8558cc4-5qlrn

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

opendatahub

kserve-module-controller-manager-7566cf5fc8-2p5lx_b836b7f9-b8c7-4152-bbad-04edbd59fdbb

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-2p5lx_b836b7f9-b8c7-4152-bbad-04edbd59fdbb became leader

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-7dxb9

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-ghltt

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

multus

authorino-57f7f8659f-sn9rq

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-57f7f8659f-sn9rq

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-ghltt

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

kubelet

authorino-56c6c9ddd5-7dxb9

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-56c6c9ddd5-7dxb9

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

multus

limitador-limitador-7c9b87d574-ghltt

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-controller-manager-67b8558cc4-5qlrn

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9zhx5

Started

Container started

openshift-jobset-operator

kubelet

jobset-controller-manager-67b8558cc4-5qlrn

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 2.716s (2.716s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-67b8558cc4-5qlrn

Started

Container started

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-ghltt

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9zhx5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 2.618s (2.618s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-ghltt

Started

Container started

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-ghltt

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 909ms (909ms including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9zhx5

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-7dxb9

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

authorino-57f7f8659f-sn9rq

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-7dxb9

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-sn9rq

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-7dxb9

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.635s (2.635s including waiting). Image size: 204096810 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-jobset-operator

jobset-controller-manager-67b8558cc4-5qlrn_e8a4023a-4612-45f0-ab0e-1283e5428445

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-67b8558cc4-5qlrn_e8a4023a-4612-45f0-ab0e-1283e5428445 became leader

kuadrant-system

kubelet

authorino-57f7f8659f-sn9rq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.313s (3.313s including waiting). Image size: 204096810 bytes.

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-sn9rq

kuadrant-system

kubelet

authorino-57f7f8659f-sn9rq

Killing

Stopping container authorino

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-9zhx5

Killing

Stopping container limitador

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-9zhx5

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-9lb82

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-9lb82

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-9lb82

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

opendatahub

kubelet

kserve-controller-manager-776cc66d47-7xw9r

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-z2nb2

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-64d68d6d56 from 0 to 1

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

multus

kserve-controller-manager-776cc66d47-7xw9r

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

replicaset-controller

maas-controller-64d68d6d56

SuccessfulCreate

Created pod: maas-controller-64d68d6d56-76wsv

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-vqsrv

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-7xw9r

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-4kw4d

opendatahub

multus

odh-model-controller-769875db56-z2nb2

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

opendatahub

multus

model-serving-api-75bd89fdfc-4kw4d

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-vqsrv

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

odh-model-controller-769875db56-z2nb2

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

multus

maas-controller-64d68d6d56-76wsv

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-64d68d6d56-76wsv

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8"

opendatahub

kubelet

model-serving-api-75bd89fdfc-4kw4d

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

replicaset-controller

maas-controller-5db8dc9fd8

SuccessfulCreate

Created pod: maas-controller-5db8dc9fd8-76ms5

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-vqsrv

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-9lb82

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.67s (2.67s including waiting). Image size: 185476477 bytes.

opendatahub

kubelet

maas-controller-5db8dc9fd8-76ms5

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8"

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-9lb82

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-9lb82

Created

Container created

opendatahub

multus

maas-controller-5db8dc9fd8-76ms5

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

opendatahub

kubelet

kserve-controller-manager-776cc66d47-7xw9r

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-t8szh

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-t8szh

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

opendatahub

kubelet

kserve-controller-manager-776cc66d47-7xw9r

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.942s (3.942s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-4kw4d

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 3.13s (3.13s including waiting). Image size: 184925761 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-4kw4d

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-4kw4d

Started

Container started

opendatahub

kubelet

kserve-controller-manager-776cc66d47-7xw9r

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-vqsrv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 4.055s (4.055s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-vqsrv

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-vqsrv

Created

Container created

opendatahub

kubelet

maas-controller-64d68d6d56-76wsv

Started

Container started

opendatahub

llmisvc-controller-manager-869d5dcc5d-vqsrv_64d3a1d4-f561-47b5-95c3-9ee153922f69

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-vqsrv_64d3a1d4-f561-47b5-95c3-9ee153922f69 became leader

opendatahub

kubelet

maas-controller-5db8dc9fd8-76ms5

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8" in 4.113s (4.113s including waiting). Image size: 200094742 bytes.

opendatahub

kubelet

maas-controller-5db8dc9fd8-76ms5

Created

Container created

opendatahub

kubelet

maas-controller-5db8dc9fd8-76ms5

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-z2nb2

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 4.933s (4.933s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-z2nb2

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-z2nb2

Started

Container started

opendatahub

kubelet

maas-controller-64d68d6d56-76wsv

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8" in 4.526s (4.526s including waiting). Image size: 200094742 bytes.

opendatahub

kubelet

maas-controller-64d68d6d56-76wsv

Created

Container created

opendatahub

odh-model-controller-769875db56-z2nb2_bd3598b5-55e4-40be-9cda-83f95ca986d9

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-z2nb2_bd3598b5-55e4-40be-9cda-83f95ca986d9 became leader

opendatahub

kserve-controller-manager-776cc66d47-7xw9r_f824da68-e3bd-472d-8b0b-dbf7994289f2

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-7xw9r_f824da68-e3bd-472d-8b0b-dbf7994289f2 became leader

opendatahub

maas-controller-64d68d6d56-76wsv_91ca8621-fad3-4acc-8428-58bf78c64c99

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-64d68d6d56-76wsv_91ca8621-fad3-4acc-8428-58bf78c64c99 became leader

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-c2cbh

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-c2cbh

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-c2cbh

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-64d68d6d56 from 1 to 0

opendatahub

kubelet

maas-controller-64d68d6d56-76wsv

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-64d68d6d56

SuccessfulDelete

Deleted pod: maas-controller-64d68d6d56-76wsv

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-c2cbh

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 10.37s (10.37s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-c2cbh

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-c2cbh

Started

Container started

opendatahub

maas-controller-5db8dc9fd8-76ms5_280b16f5-dbd9-4d7a-91b4-5612984d5df6

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-5db8dc9fd8-76ms5_280b16f5-dbd9-4d7a-91b4-5612984d5df6 became leader
(x6)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-hv5z8

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-hv5z8

kuadrant-system

kubelet

authorino-c75459655-fdx74

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-6f6944bcdd-hv5z8

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-c75459655-fdx74

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Created

Container created

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-c75459655 from 0 to 1

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulDelete

Deleted pod: authorino-6889cbd8c4-7696f

kuadrant-system

multus

authorino-6f6944bcdd-hv5z8

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulCreate

Created pod: authorino-6889cbd8c4-7696f

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

replicaset-controller

authorino-c75459655

SuccessfulCreate

Created pod: authorino-c75459655-fdx74
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6889cbd8c4 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6889cbd8c4 from 1 to 0

kuadrant-system

kubelet

authorino-c75459655-fdx74

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.628s (2.628s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6f6944bcdd-hv5z8

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.861s (2.861s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-c75459655-fdx74

Started

Container started

kuadrant-system

kubelet

authorino-c75459655-fdx74

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-hv5z8

Created

Container created

kuadrant-system

kubelet

authorino-6f6944bcdd-hv5z8

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-7dxb9

kuadrant-system

kubelet

authorino-56c6c9ddd5-7dxb9

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-6f6944bcdd-hv5z8

Killing

Stopping container authorino

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-fttqq

Started

Container started

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-fttqq

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-6466c554d8-fttqq

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-fttqq

Created

Container created

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-fttqq

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-fttqq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 268ms (268ms including waiting). Image size: 235100981 bytes.
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Unhealthy

Startup probe failed: Get "http://10.134.0.20:15021/healthz/ready": dial tcp 10.134.0.20:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

ProbeError

Startup probe error: Get "http://10.134.0.20:15021/healthz/ready": dial tcp 10.134.0.20:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-vqsrv

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-vqsrv

Killing

Stopping container manager

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-xjj8d

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-xjj8d

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

multus

ai-gateway-controller-5d8dddb676-xjj8d

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-xjj8d

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-xjj8d

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 1.534s (1.534s including waiting). Image size: 183979430 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-xjj8d

Created

Container created

opendatahub

ai-gateway-controller-5d8dddb676-xjj8d_f7918a4b-4983-4cb7-a0e1-fc0263205efe

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-xjj8d_f7918a4b-4983-4cb7-a0e1-fc0263205efe became leader

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-2btkh

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulCreate

Created pod: payload-processing-55fbbbf9dc-rx5hk
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-pre-processing-7b98bb678c-2btkh

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-2btkh

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

FailedCreate

Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-processing-55fbbbf9dc-rx5hk

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rx5hk

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rx5hk

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-2btkh

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-2btkh

Started

Container started

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rx5hk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-2btkh

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.362s (2.362s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rx5hk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.223s (2.223s including waiting). Image size: 125353963 bytes.

opendatahub

replicaset-controller

ai-gateway-controller-6cc7b895d5

SuccessfulCreate

Created pod: ai-gateway-controller-6cc7b895d5-5ldn8

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-6cc7b895d5 from 0 to 1

opendatahub

multus

ai-gateway-controller-6cc7b895d5-5ldn8

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-6cc7b895d5-5ldn8

Started

Container started

opendatahub

kubelet

ai-gateway-controller-6cc7b895d5-5ldn8

Created

Container created

opendatahub

kubelet

ai-gateway-controller-6cc7b895d5-5ldn8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 435ms (435ms including waiting). Image size: 183979430 bytes.

opendatahub

kubelet

ai-gateway-controller-6cc7b895d5-5ldn8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

llmisvc-controller-manager-6466c554d8-fttqq_60c17352-fddd-4c41-b818-f302b6c7ab91

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-fttqq_60c17352-fddd-4c41-b818-f302b6c7ab91 became leader

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-xjj8d

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-xjj8d

opendatahub

ai-gateway-controller-6cc7b895d5-5ldn8_8740b7de-b683-4684-9ebe-f5798c49f482

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-6cc7b895d5-5ldn8_8740b7de-b683-4684-9ebe-f5798c49f482 became leader

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg

Created

Container created

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-hlvsg

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.262s (1.262s including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-kwrdx

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-kwrdx

Started

Container started

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-kwrdx

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-kwrdx

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-kwrdx

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 203ms (203ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-kwrdx

Created

Container created

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-4zm5n

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4zm5n

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-4zm5n

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4zm5n

Started

Container started

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4zm5n

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.174s (1.174s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-4zm5n

Created

Container created

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-nnfxn

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-nnfxn

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-nnfxn

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-nnfxn

Started

Container started

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-nnfxn

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-nnfxn

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 192ms (192ms including waiting). Image size: 98346788 bytes.

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 226ms (226ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8454nw

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-92h45

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

multus

echo-simulator-kserve-64b6f8f96c-92h45

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

llm

kubelet

echo-simulator-kserve-64b6f8f96c-92h45

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 237ms (237ms including waiting). Image size: 98346788 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-92h45

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-cf2lf

Started

Container started

llm

kubelet

echo-simulator-kserve-64b6f8f96c-92h45

Started

Container started

llm

kubelet

echo-simulator-kserve-64b6f8f96c-92h45

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 1.815s (1.815s including waiting). Image size: 78208878 bytes.

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-8q5hv

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 1 to 2

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-8q5hv

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 223ms (223ms including waiting). Image size: 98346788 bytes.

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Started

Container started
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.806s (2.806s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

Created

Container created

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

ProbeError

Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

Unhealthy

Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-b8n6c

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-97tgx

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-b8n6c

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-97tgx

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 2 to 4

kuadrant-system

replicaset-controller

authorino-64c547bcd4

SuccessfulCreate

Created pod: authorino-64c547bcd4-s7w4p

kuadrant-system

kubelet

authorino-64c547bcd4-s7w4p

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-64c547bcd4-s7w4p

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

Started

Container started

kuadrant-system

replicaset-controller

authorino-c75459655

SuccessfulDelete

Deleted pod: authorino-c75459655-fdx74

kuadrant-system

kubelet

authorino-c75459655-fdx74

Killing

Stopping container authorino
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-c75459655 from 1 to 0

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.693s (2.693s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

Created

Container created

kuadrant-system

kubelet

authorino-64c547bcd4-s7w4p

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 514ms (514ms including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-64c547bcd4-s7w4p

Created

Container created

kuadrant-system

kubelet

authorino-64c547bcd4-s7w4p

Started

Container started
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

Unhealthy

Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

ProbeError

Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-pxflc

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-9pqlk

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-259d8

Created

Container created

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 4 to 8

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason:

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-4nbcz

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

Created

Container created

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-9pqlk

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-4nbcz

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-259d8

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-259d8

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-259d8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-259d8

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-9pqlk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-pxflc

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-9pqlk

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-9pqlk

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

Started

Container started
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-9pqlk

ProbeError

Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-259d8

ProbeError

Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-259d8

Unhealthy

Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

Unhealthy

Startup probe failed: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

ProbeError

Startup probe error: Get "http://10.134.0.45:15021/healthz/ready": dial tcp 10.134.0.45:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-9pqlk

Unhealthy

Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-jbt2l

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 8 to 10

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-bxtwk

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-jbt2l

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-55fbd65575-bxtwk

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Created

Container created
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

ProbeError

Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

Unhealthy

Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-4be645

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-4be645

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-555d1e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-555d1e

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-c0c28f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-c0c28f

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 0 to 1

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-c0c28f-7854685bb7

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-c0c28f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-c0c28f: serviceaccount "payload-processing-e2e-worker-w4-c0c28f" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-c0c28f-7854685bb7

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-c0c28f-7854685bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-c0c28f: serviceaccount "payload-processing-e2e-worker-w4-c0c28f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-c0c28f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-c0c28f-7854685bb7 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-4be645-544cb69f75

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-4be645: serviceaccount "payload-processing-e2e-worker-w2-4be645" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-4be645-544cb69f75

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-4be645-544cb69f75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-4be645: serviceaccount "payload-processing-e2e-worker-w2-4be645" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-4be645

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-4be645-58d4569f87 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-4be645

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-4be645-544cb69f75 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-555d1e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-555d1e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-555d1e-758c9888f from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-555d1e-758c9888f

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-555d1e-758c9888f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-555d1e: serviceaccount "payload-processing-e2e-worker-w0-555d1e" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-555d1e-758c9888f

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-555d1e: serviceaccount "payload-processing-e2e-worker-w0-555d1e" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.356s (3.356s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.48s (3.48s including waiting). Image size: 125353963 bytes.

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 770ms (770ms including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.901s (1.901s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.989s (1.989s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

Unhealthy

Readiness probe failed: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

ProbeError

Readiness probe error: Get "http://10.134.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

Unhealthy

Readiness probe failed: Get "http://10.134.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

ProbeError

Readiness probe error: Get "http://10.134.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

Unhealthy

Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

Unhealthy

Readiness probe failed: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

ProbeError

Readiness probe error: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

ProbeError

Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

Unhealthy

Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

ProbeError

Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

ProbeError

Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

Unhealthy

Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-inf-a-758617

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-758617

openshift-ingress

multus

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-inf-a-758617-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-758617-openshift-default-754657d5bc from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-a-758617-openshift-default-754657d5bc

SuccessfulCreate

Created pod: e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-758617

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-758617-786b6d75ff from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-758617-786b6d75ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-758617

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-758617-84bfb45777 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-758617-84bfb45777

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-758617-84bfb45777

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-758617-84bfb45777-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-758617: serviceaccount "payload-processing-e2e-inf-a-758617" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-758617-786b6d75ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-758617-786b6d75ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-758617: serviceaccount "payload-processing-e2e-inf-a-758617" not found

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

Started

Container started

openshift-ingress

multus

payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-758617-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-758617-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj
(x5)

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

Unhealthy

Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

ProbeError

Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused body:

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

Unhealthy

Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

ProbeError

Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-inf-b-854b9e-openshift-default-7dd9bd658c

SuccessfulCreate

Created pod: e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

openshift-ingress

deployment-controller

e2e-inf-b-854b9e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-854b9e-openshift-default-7dd9bd658c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-b-854b9e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-854b9e

openshift-ingress

multus

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

Created

Container created

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-758617-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-758617-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-854b9e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

ProbeError

Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

Unhealthy

Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-854b9e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-854b9e-56687854c7

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-854b9e-56687854c7

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-854b9e-56687854c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-854b9e: serviceaccount "payload-processing-e2e-inf-b-854b9e" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-854b9e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-854b9e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-854b9e-56687854c7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-854b9e: serviceaccount "payload-processing-e2e-inf-b-854b9e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

openshift-ingress

kubelet

payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

Started

Container started

openshift-ingress

multus

payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused body:

opendatahub

kubelet

test-curl-7807-e92a4a

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7807-e92a4a

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 2 to 4

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

Started

Container started

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

Started

Container started

opendatahub

kubelet

test-curl-7807-e92a4a

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

Created

Container created

opendatahub

kubelet

test-curl-7807-e92a4a

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7807-e92a4a

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.384s (2.384s including waiting). Image size: 106100986 bytes.

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7807-2c2ad3

Started

Container started

opendatahub

kubelet

test-curl-7807-e92a4a

Killing

Stopping container test-curl-7807-e92a4a

opendatahub

kubelet

test-curl-7807-2c2ad3

Created

Container created

opendatahub

multus

test-curl-7807-2c2ad3

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7807-2c2ad3

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7807-2c2ad3

Killing

Stopping container test-curl-7807-2c2ad3
(x6)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-854b9e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-854b9e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-a-bf0b0d-openshift-default-697867ff97

SuccessfulCreate

Created pod: e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

openshift-ingress

deployment-controller

e2e-extmod-a-bf0b0d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-bf0b0d-openshift-default-697867ff97 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-bf0b0d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-bf0b0d
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

Unhealthy

Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

ProbeError

Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused body:
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-bf0b0d: serviceaccount "payload-processing-e2e-extmod-a-bf0b0d" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-bf0b0d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-bf0b0d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-bf0b0d: serviceaccount "payload-processing-e2e-extmod-a-bf0b0d" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 2 to 4

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

ProbeError

Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

Unhealthy

Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

ProbeError

Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f

SuccessfulCreate

Created pod: e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

openshift-ingress

deployment-controller

e2e-extmod-b-2bb4ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-2bb4ef

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-2bb4ef

openshift-ingress

multus

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 4 to 6

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

Started

Container started

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-2bb4ef

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-2bb4ef: serviceaccount "payload-processing-e2e-extmod-b-2bb4ef" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-2bb4ef: serviceaccount "payload-processing-e2e-extmod-b-2bb4ef" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-2bb4ef

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9 from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-bf0b0d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

ProbeError

Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-bf0b0d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

Unhealthy

Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

ProbeError

Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

Unhealthy

Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 4 to 8

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

Started

Container started

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 2 to 4

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

Created

Container created

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-bf0b0d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-bf0b0d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

ProbeError

Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

Unhealthy

Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

Unhealthy

Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

ProbeError

Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

ProbeError

Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

Unhealthy

Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

Unhealthy

Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

ProbeError

Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

ProbeError

Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

Unhealthy

Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x6)

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

ProbeError

Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

Unhealthy

Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 8 to 10

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

Created

Container created

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

AddedInterface

Add eth0 [10.133.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

Started

Container started

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

Started

Container started

openshift-ingress

multus

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 5; reason:

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 4 to 5

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 6 to 10

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

Unhealthy

Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

ProbeError

Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

Unhealthy

Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

ProbeError

Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

Unhealthy

Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

Unhealthy

Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

ProbeError

Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

ProbeError

Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

ProbeError

Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

Unhealthy

Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2bb4ef-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2bb4ef-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-praxis-a-de8cc7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-a-de8cc7

openshift-ingress

deployment-controller

e2e-praxis-a-de8cc7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-a-de8cc7-openshift-default-6486dc96d5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5

SuccessfulCreate

Created pod: e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

openshift-ingress

multus

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

AddedInterface

Add eth0 [10.133.0.78/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-a-de8cc7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-a-de8cc7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-a-de8cc7-d8b8475f9 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-de8cc7: serviceaccount "payload-processing-e2e-praxis-a-de8cc7" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-de8cc7: serviceaccount "payload-processing-e2e-praxis-a-de8cc7" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt

Created

Container created

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

AddedInterface

Add eth0 [10.133.0.85/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 5 to 10

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

AddedInterface

Add eth0 [10.133.0.84/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

Killing

Stopping container payload-pre-processing
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

ProbeError

Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

Unhealthy

Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

Unhealthy

Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

ProbeError

Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

Unhealthy

Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

ProbeError

Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

Started

Container started

openshift-ingress

deployment-controller

e2e-praxis-b-514524-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-b-514524-openshift-default-5b86c4994f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-praxis-b-514524

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-b-514524

openshift-ingress

replicaset-controller

e2e-praxis-b-514524-openshift-default-5b86c4994f

SuccessfulCreate

Created pod: e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

openshift-ingress

multus

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

Unhealthy

Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

ProbeError

Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-758617-786b6d75ff-r4sx7

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-a-758617-84bfb45777-wtq4h

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-inf-a-758617-openshift-default-754657d5bc-vcf88

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-de8cc7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-de8cc7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-514524-75468d84dc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-b-514524-75468d84dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-514524: serviceaccount "payload-processing-e2e-praxis-b-514524" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-514524-75468d84dc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-514524-5cd594dcc9

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-b-514524-5cd594dcc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-514524: serviceaccount "payload-processing-e2e-praxis-b-514524" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-514524-5cd594dcc9

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-b-514524

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-b-514524-5cd594dcc9 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-b-514524

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-b-514524-75468d84dc from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk

Started

Container started

openshift-ingress

multus

payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-rdlmf

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts

Killing

Stopping container payload-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-de8cc7-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-de8cc7-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x4)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-866159

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-866159

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 0 to 1

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

AddedInterface

Add eth0 [10.134.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-inf-b-854b9e-openshift-default-7dd9bd658c-5dg9r

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-b-854b9e-56687854c7-2m9ts

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-854b9e-75c9b5dcdc-h8j4b

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-866159-7ff65f964c

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-866159-7ff65f964c

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-866159-7ff65f964c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-866159: serviceaccount "payload-processing-e2e-worker-w3-866159" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-866159

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-866159: serviceaccount "payload-processing-e2e-worker-w3-866159" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-866159

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-866159-7ff65f964c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

AddedInterface

Add eth0 [10.134.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

Started

Container started

openshift-ingress

multus

payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

Unhealthy

Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

ProbeError

Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-b8n6c

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-clxsz

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-b8n6c

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-bxtwk

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-4nbcz

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4nbcz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-bxtwk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-clxsz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-514524-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-514524-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

ProbeError

Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

Unhealthy

Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-514524-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-514524-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Unhealthy

Readiness probe failed: Get "https://10.132.0.44:8000/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Unhealthy

Readiness probe failed: Get "https://10.132.0.44:8000/ready": context deadline exceeded

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2bb4ef-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-extmod-b-2bb4ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f from 1 to 2

openshift-ingress

replicaset-controller

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f

SuccessfulCreate

Created pod: e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

openshift-ingress

multus

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

AddedInterface

Add eth0 [10.134.0.97/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

AddedInterface

Add eth0 [10.133.0.109/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

Created

Container created

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

ProbeError

Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

Unhealthy

Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

Unhealthy

Readiness probe failed: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

ProbeError

Readiness probe error: Get "http://10.133.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

ProbeError

Readiness probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

Unhealthy

Readiness probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

Unhealthy

Readiness probe failed: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

ProbeError

Readiness probe error: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

ProbeError

Readiness probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

Unhealthy

Readiness probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

ProbeError

Startup probe error: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

Unhealthy

Startup probe failed: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-9ba664

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-9ba664

openshift-ingress

deployment-controller

e2e-extmod-a-9ba664-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-9ba664-openshift-default-5b57547dc7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-9ba664-openshift-default-5b57547dc7

SuccessfulCreate

Created pod: e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

openshift-ingress

multus

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

AddedInterface

Add eth0 [10.134.0.99/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-de8cc7-d8b8475f9-227pt

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-9ba664

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-9ba664-54cdd94959 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-9ba664: serviceaccount "payload-processing-e2e-extmod-a-9ba664" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-9ba664-54cdd94959

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-9ba664-54cdd94959

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-9ba664-54cdd94959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-9ba664: serviceaccount "payload-processing-e2e-extmod-a-9ba664" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-9ba664

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

AddedInterface

Add eth0 [10.134.0.101/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-de8cc7-57d6dfbcbc-c4jsb

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f

SuccessfulCreate

Created pod: e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w4-f1e1c2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-f1e1c2

openshift-ingress

deployment-controller

e2e-shared-a-w4-f1e1c2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f from 0 to 1

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

AddedInterface

Add eth0 [10.133.0.116/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

AddedInterface

Add eth0 [10.133.0.115/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 3 to 6

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

Started

Container started

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

AddedInterface

Add eth0 [10.134.0.104/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-praxis-a-de8cc7-openshift-default-6486dc96d5-b99sm

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-f1e1c2: serviceaccount "payload-processing-e2e-shared-a-w4-f1e1c2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w4-f1e1c2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w4-f1e1c2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-f1e1c2: serviceaccount "payload-processing-e2e-shared-a-w4-f1e1c2" not found
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

ProbeError

Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused body:

openshift-ingress

multus

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t

AddedInterface

Add eth0 [10.134.0.107/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr

AddedInterface

Add eth0 [10.134.0.106/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

ProbeError

Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

Unhealthy

Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

Unhealthy

Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

Unhealthy

Startup probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

ProbeError

Startup probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9ba664-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9ba664-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-extmod-b-12b8da-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-12b8da-openshift-default-7df679ddc7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-12b8da-openshift-default-7df679ddc7

SuccessfulCreate

Created pod: e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-12b8da

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-12b8da

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

AddedInterface

Add eth0 [10.133.0.118/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 6 to 7

openshift-ingress

multus

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

AddedInterface

Add eth0 [10.134.0.108/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

Unhealthy

Startup probe failed: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

ProbeError

Startup probe error: Get "http://10.134.0.99:15021/healthz/ready": dial tcp 10.134.0.99:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

Unhealthy

Readiness probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-12b8da-7496c4f78f

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-12b8da

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-12b8da-7496c4f78f

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-12b8da-7496c4f78f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-12b8da: serviceaccount "payload-processing-e2e-extmod-b-12b8da" not found
(x2)

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

ProbeError

Readiness probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-12b8da

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-12b8da-7496c4f78f from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-12b8da: serviceaccount "payload-processing-e2e-extmod-b-12b8da" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw

AddedInterface

Add eth0 [10.134.0.110/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

AddedInterface

Add eth0 [10.134.0.111/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

Unhealthy

Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

ProbeError

Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-514524-5cd594dcc9-hctph

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-514524-75468d84dc-ffggk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-praxis-b-514524-openshift-default-5b86c4994f-jdzkf

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 10 to 6

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-jxt77

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-fqjwq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-86vvc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-x2w9d

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9ba664-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-9ba664-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-12b8da-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-12b8da-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-shared-b-w4-623b4b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47

SuccessfulCreate

Created pod: e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-f1e1c2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 7 to 10

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-f1e1c2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w4-623b4b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-623b4b

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

Created

Container created

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

AddedInterface

Add eth0 [10.134.0.113/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

Started

Container started

openshift-ingress

multus

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

AddedInterface

Add eth0 [10.134.0.114/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

AddedInterface

Add eth0 [10.133.0.121/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

Unhealthy

Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

ProbeError

Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w4-623b4b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w4-623b4b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c from 0 to 1
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

Unhealthy

Startup probe failed: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

Unhealthy

Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

ProbeError

Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-623b4b: serviceaccount "payload-processing-e2e-shared-b-w4-623b4b" not found
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

ProbeError

Startup probe error: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

ProbeError

Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

Unhealthy

Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-623b4b: serviceaccount "payload-processing-e2e-shared-b-w4-623b4b" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

AddedInterface

Add eth0 [10.133.0.122/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

AddedInterface

Add eth0 [10.133.0.123/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.823s (2.823s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.886s (2.886s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

ProbeError

Readiness probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

Unhealthy

Readiness probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

Unhealthy

Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

ProbeError

Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zvq6k

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-cknl5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-zdbqj

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-cknl5

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-cknl5

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zvq6k

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-cknl5

AddedInterface

Add eth0 [10.133.0.128/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-zdbqj

AddedInterface

Add eth0 [10.134.0.118/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-89nq9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-89nq9

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-89nq9

AddedInterface

Add eth0 [10.134.0.117/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-zvq6k

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-89nq9

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-zvq6k

AddedInterface

Add eth0 [10.134.0.116/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-cknl5

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zvq6k

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-89nq9

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-f1e1c2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-f1e1c2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-cknl5

Unhealthy

Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-cknl5

ProbeError

Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

ProbeError

Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

Unhealthy

Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zvq6k

ProbeError

Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused body:
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zvq6k

Unhealthy

Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-89nq9

Unhealthy

Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-89nq9

ProbeError

Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

Unhealthy

Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

ProbeError

Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-12b8da-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-12b8da-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

ProbeError

Readiness probe error: Get "http://10.134.0.114:15021/healthz/ready": read tcp 10.134.0.2:36412->10.134.0.114:15021: read: connection reset by peer body:

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

Unhealthy

Readiness probe failed: Get "http://10.134.0.114:15021/healthz/ready": read tcp 10.134.0.2:36412->10.134.0.114:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

Unhealthy

Startup probe failed: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

ProbeError

Startup probe error: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-623b4b-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-623b4b-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

models-as-a-service

replicaset-controller

payload-processing-external-model-8569d75f4b

SuccessfulCreate

Created pod: payload-processing-external-model-8569d75f4b-b6ftn
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-8569d75f4b

FailedCreate

Error creating: pods "payload-processing-external-model-8569d75f4b-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-8569d75f4b from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-8569d75f4b-b6ftn

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-8569d75f4b-b6ftn

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-8569d75f4b-b6ftn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-8569d75f4b-b6ftn

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-623b4b-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-623b4b-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 6 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-7nfbl

openshift-ingress

deployment-controller

e2e-ait-c9fc7d82-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-c9fc7d82-openshift-default-64bfc7885c from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-c9fc7d82-openshift-default-64bfc7885c

SuccessfulCreate

Created pod: e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

openshift-ingress

gateway_labeler_controller

e2e-ait-c9fc7d82

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c9fc7d82

openshift-ingress

multus

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

AddedInterface

Add eth0 [10.133.0.139/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-c9fc7d82-74c8ff8767

FailedCreate

Error creating: pods "payload-processing-e2e-ait-c9fc7d82-74c8ff8767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c9fc7d82: serviceaccount "payload-processing-e2e-ait-c9fc7d82" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c9fc7d82: serviceaccount "payload-processing-e2e-ait-c9fc7d82" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-c9fc7d82

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-c9fc7d82

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-c9fc7d82-74c8ff8767 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-c9fc7d82-74c8ff8767

SuccessfulCreate

Created pod: payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz

openshift-ingress

kubelet

payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz

AddedInterface

Add eth0 [10.132.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz

AddedInterface

Add eth0 [10.132.0.83/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c9fc7d82-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c9fc7d82-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-259d8

ProbeError

Readiness probe error: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-259d8

Unhealthy

Readiness probe failed: Get "http://10.132.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

Unhealthy

Startup probe failed: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

ProbeError

Startup probe error: Get "http://10.133.0.139:15021/healthz/ready": dial tcp 10.133.0.139:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c9fc7d82-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c9fc7d82-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c9fc7d82-7b99bdf5dc-dqsrz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-c9fc7d82-74c8ff8767-k2mnz

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 10 to 6

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ntc7d

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfst5

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-ngbv5

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-zbf79

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-migrate-87a1a06b-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-87a1a06b-gw

models-as-a-service

kubelet

payload-processing-external-model-8569d75f4b-b6ftn

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-migrate-87a1a06b-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7

SuccessfulCreate

Created pod: e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

openshift-ingress

multus

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

AddedInterface

Add eth0 [10.133.0.143/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-ait-c9fc7d82-openshift-default-64bfc7885c-sj7kd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

opendatahub

kubelet

test-curl-7816-a964bc

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

multus

test-curl-7816-a964bc

AddedInterface

Add eth0 [10.133.0.144/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7816-a964bc

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-87a1a06b-74cbdbd569

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-87a1a06b-74cbdbd569

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-87a1a06b-74cbdbd569-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-87a1a06b: serviceaccount "payload-processing-e2e-migrate-87a1a06b" not found

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-87a1a06b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-87a1a06b-74cbdbd569 from 0 to 1

opendatahub

kubelet

test-curl-7816-a964bc

Created

Container created

opendatahub

kubelet

test-curl-7816-a964bc

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 905ms (905ms including waiting). Image size: 106100986 bytes.
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-87a1a06b: serviceaccount "payload-processing-e2e-migrate-87a1a06b" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-87a1a06b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth

openshift-ingress

multus

payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95

AddedInterface

Add eth0 [10.133.0.146/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth

AddedInterface

Add eth0 [10.133.0.145/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95

Started

Container started

opendatahub

kubelet

test-curl-7816-a964bc

Killing

Stopping container test-curl-7816-a964bc
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-85d6f9ccbb

FailedCreate

Error creating: pods "payload-processing-external-model-85d6f9ccbb-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-85d6f9ccbb

SuccessfulCreate

Created pod: payload-processing-external-model-85d6f9ccbb-86w9p

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-85d6f9ccbb from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-86w9p

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-86w9p

Created

Container created

models-as-a-service

multus

payload-processing-external-model-85d6f9ccbb-86w9p

AddedInterface

Add eth0 [10.133.0.149/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-86w9p

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-86w9p

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-85d6f9ccbb from 0 to 1
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-85d6f9ccbb

FailedCreate

Error creating: pods "payload-processing-external-model-85d6f9ccbb-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-85d6f9ccbb

SuccessfulCreate

Created pod: payload-processing-external-model-85d6f9ccbb-pssz5

models-as-a-service

multus

payload-processing-external-model-85d6f9ccbb-pssz5

AddedInterface

Add eth0 [10.133.0.152/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-pssz5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-pssz5

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-pssz5

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-87a1a06b-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-87a1a06b-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-623b4b-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47

SuccessfulCreate

Created pod: e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

openshift-ingress

deployment-controller

e2e-shared-b-w4-623b4b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47 from 1 to 2

models-as-a-service

kubelet

payload-processing-external-model-85d6f9ccbb-pssz5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

Created

Container created

openshift-ingress

multus

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

AddedInterface

Add eth0 [10.133.0.153/23] from ovn-kubernetes
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-7c999f66

FailedCreate

Error creating: pods "payload-processing-external-model-7c999f66-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-7c999f66

SuccessfulCreate

Created pod: payload-processing-external-model-7c999f66-ks4pr

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7c999f66 from 0 to 1
(x2)

models-as-a-service

kubelet

payload-processing-external-model-7c999f66-ks4pr

FailedMount

MountVolume.SetUp failed for volume "kube-api-access-699pv" : failed to fetch token: serviceaccounts "payload-processing-external-model" not found
(x2)

models-as-a-service

kubelet

payload-processing-external-model-7c999f66-ks4pr

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-de08474d6f06" not found
(x2)

models-as-a-service

kubelet

payload-processing-external-model-7c999f66-ks4pr

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-external-model-plugins" not found
(x7)

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

Unhealthy

Startup probe failed: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

ProbeError

Startup probe error: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

ProbeError

Startup probe error: Get "http://10.133.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

Unhealthy

Startup probe failed: Get "http://10.133.0.153:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4fmpg

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 5 to 4

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 6 to 5

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-z4nx8

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

Unhealthy

Readiness probe failed: Get "http://10.133.0.143:15021/healthz/ready": read tcp 10.133.0.2:33350->10.133.0.143:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

ProbeError

Readiness probe error: Get "http://10.133.0.143:15021/healthz/ready": read tcp 10.133.0.2:33350->10.133.0.143:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-extmod-a-944506-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-944506-openshift-default-855597c485 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-944506

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-944506

openshift-ingress

replicaset-controller

e2e-extmod-a-944506-openshift-default-855597c485

SuccessfulCreate

Created pod: e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

openshift-ingress

multus

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

AddedInterface

Add eth0 [10.133.0.154/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

Unhealthy

Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

ProbeError

Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-944506

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-944506-5bcfb5fb67 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-944506-5bcfb5fb67

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-944506-5bcfb5fb67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-944506: serviceaccount "payload-processing-e2e-extmod-a-944506" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-944506

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-944506-786f89d5dc from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-944506: serviceaccount "payload-processing-e2e-extmod-a-944506" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 4 to 3

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

Started

Container started

openshift-ingress

multus

payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-944506-5bcfb5fb67

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-qtlwk

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 10 to 9

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 9; reason: All metrics below target

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

AddedInterface

Add eth0 [10.132.0.91/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-4jgf4

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-b-12b8da-openshift-default-7df679ddc7

SuccessfulCreate

Created pod: e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

openshift-ingress

deployment-controller

e2e-extmod-b-12b8da-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-12b8da-openshift-default-7df679ddc7 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-12b8da-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-87a1a06b-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-87a1a06b-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

AddedInterface

Add eth0 [10.133.0.156/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-87a1a06b-765c4f5755-cgxth

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-87a1a06b-74cbdbd569-mbn95

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

ProbeError

Startup probe error: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

Unhealthy

Startup probe failed: Get "http://10.133.0.156:15021/healthz/ready": dial tcp 10.133.0.156:15021: connect: connection refused

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-747f596d58 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-747f596d58-lncqr

Started

Container started
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-747f596d58

FailedCreate

Error creating: pods "payload-processing-external-model-747f596d58-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

kubelet

payload-processing-external-model-747f596d58-lncqr

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-747f596d58-lncqr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-747f596d58

SuccessfulCreate

Created pod: payload-processing-external-model-747f596d58-lncqr

models-as-a-service

multus

payload-processing-external-model-747f596d58-lncqr

AddedInterface

Add eth0 [10.132.0.93/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-zbsqg

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 9 to 6

openshift-ingress

kubelet

e2e-migrate-87a1a06b-gw-openshift-default-bd976cdf7-7bkw8

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-wt9kx

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-hwlcv

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ait-8416e2e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-8416e2e2-openshift-default-677756d6f9 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-8416e2e2-openshift-default-677756d6f9

SuccessfulCreate

Created pod: e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

openshift-ingress

gateway_labeler_controller

e2e-ait-8416e2e2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8416e2e2

openshift-ingress

multus

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

AddedInterface

Add eth0 [10.132.0.94/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-8416e2e2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-8416e2e2-75fd84c66f from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8416e2e2-75fd84c66f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-8416e2e2-75fd84c66f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8416e2e2: serviceaccount "payload-processing-e2e-ait-8416e2e2" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8416e2e2: serviceaccount "payload-processing-e2e-ait-8416e2e2" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-8416e2e2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8416e2e2-75fd84c66f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng

openshift-ingress

kubelet

payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng

AddedInterface

Add eth0 [10.132.0.97/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8

AddedInterface

Add eth0 [10.132.0.96/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng

Started

Container started

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 5 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-5k5q4

openshift-ingress

deployment-controller

e2e-extmod-b-673ba8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-673ba8-openshift-default-755484574f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-673ba8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-673ba8

openshift-ingress

replicaset-controller

e2e-extmod-b-673ba8-openshift-default-755484574f

SuccessfulCreate

Created pod: e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

openshift-ingress

multus

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

AddedInterface

Add eth0 [10.133.0.157/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-54cc564d84

SuccessfulCreate

Created pod: payload-processing-external-model-54cc564d84-2md8z

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-54cc564d84 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-54cc564d84-2md8z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-54cc564d84-2md8z

AddedInterface

Add eth0 [10.133.0.158/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-54cc564d84-2md8z

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-54cc564d84-2md8z

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-673ba8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999 from 0 to 1

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Unhealthy

Readiness probe failed: Get "http://10.133.0.154:15021/healthz/ready": read tcp 10.133.0.2:36124->10.133.0.154:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

ProbeError

Readiness probe error: Get "http://10.133.0.154:15021/healthz/ready": read tcp 10.133.0.2:36124->10.133.0.154:15021: read: connection reset by peer body:
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-673ba8-7664594f7f

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-673ba8-7664594f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-673ba8: serviceaccount "payload-processing-e2e-extmod-b-673ba8" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-673ba8-7664594f7f

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-673ba8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-673ba8-7664594f7f from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-673ba8: serviceaccount "payload-processing-e2e-extmod-b-673ba8" not found
(x2)

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb

AddedInterface

Add eth0 [10.134.0.124/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk

AddedInterface

Add eth0 [10.134.0.123/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

ProbeError

Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Unhealthy

Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused

models-as-a-service

kubelet

payload-processing-external-model-747f596d58-lncqr

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-747f596d58

SuccessfulDelete

Deleted pod: payload-processing-external-model-747f596d58-lncqr

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-747f596d58 from 1 to 0

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-673ba8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-673ba8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x7)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-g65gv

Unhealthy

Readiness probe failed: Get "https://10.132.0.44:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-944506-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-944506-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

Started

Container started

models-as-a-service

replicaset-controller

payload-processing-external-model-5c4dd54948

SuccessfulCreate

Created pod: payload-processing-external-model-5c4dd54948-rb9gr

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-rb9gr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-rb9gr

Started

Container started

models-as-a-service

multus

payload-processing-external-model-5c4dd54948-rb9gr

AddedInterface

Add eth0 [10.132.0.98/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-rb9gr

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8416e2e2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8416e2e2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

Unhealthy

Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

ProbeError

Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8416e2e2-6d96d9fd4c-lf5b8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-8416e2e2-75fd84c66f-fxbng

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-54cc564d84 from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-54cc564d84-2md8z

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-54cc564d84

SuccessfulDelete

Deleted pod: payload-processing-external-model-54cc564d84-2md8z
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8416e2e2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8416e2e2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-ait-8416e2e2-openshift-default-677756d6f9-t52kj

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ait-b3690233

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-b3690233

openshift-ingress

deployment-controller

e2e-ait-b3690233-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-b3690233-openshift-default-7bcfb888c7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-b3690233-openshift-default-7bcfb888c7

SuccessfulCreate

Created pod: e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

openshift-ingress

multus

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

AddedInterface

Add eth0 [10.133.0.160/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-944506-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-944506-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

models-as-a-service

replicaset-controller

payload-processing-external-model-778c654c9

SuccessfulCreate

Created pod: payload-processing-external-model-778c654c9-lrkxv

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-778c654c9 from 0 to 1

models-as-a-service

multus

payload-processing-external-model-778c654c9-lrkxv

AddedInterface

Add eth0 [10.132.0.99/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b3690233-d5d6b4bbc

FailedCreate

Error creating: pods "payload-processing-e2e-ait-b3690233-d5d6b4bbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b3690233: serviceaccount "payload-processing-e2e-ait-b3690233" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-b3690233

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-b3690233-6f6c5cd679 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-778c654c9-lrkxv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-processing-e2e-ait-b3690233

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-b3690233-d5d6b4bbc from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-778c654c9-lrkxv

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-778c654c9-lrkxv

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b3690233-d5d6b4bbc

SuccessfulCreate

Created pod: payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b3690233: serviceaccount "payload-processing-e2e-ait-b3690233" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h

AddedInterface

Add eth0 [10.132.0.101/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f

AddedInterface

Add eth0 [10.132.0.100/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-klpjt
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 7; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 10 to 7

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5fqt8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-p87xb

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-b3690233-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-b3690233-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

Unhealthy

Readiness probe failed: Get "http://10.133.0.157:15021/healthz/ready": read tcp 10.133.0.2:53468->10.133.0.157:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

ProbeError

Readiness probe error: Get "http://10.133.0.157:15021/healthz/ready": read tcp 10.133.0.2:53468->10.133.0.157:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-5c4dd54948

SuccessfulDelete

Deleted pod: payload-processing-external-model-5c4dd54948-rb9gr

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-5c4dd54948 from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-rb9gr

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

ProbeError

Startup probe error: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

Unhealthy

Startup probe failed: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 7 to 6

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-w84bb
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-673ba8-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-673ba8-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5c4dd54948 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-5c4dd54948

SuccessfulCreate

Created pod: payload-processing-external-model-5c4dd54948-xjxbh

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-xjxbh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-xjxbh

Started

Container started

models-as-a-service

multus

payload-processing-external-model-5c4dd54948-xjxbh

AddedInterface

Add eth0 [10.134.0.126/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-xjxbh

Created

Container created

models-as-a-service

replicaset-controller

payload-processing-external-model-778c654c9

SuccessfulDelete

Deleted pod: payload-processing-external-model-778c654c9-lrkxv

models-as-a-service

kubelet

payload-processing-external-model-778c654c9-lrkxv

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-778c654c9 from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-5c4dd54948-xjxbh

Killing

Stopping container payload-processing

opendatahub

kubelet

test-curl-7816-469647

Created

Container created

opendatahub

kubelet

test-curl-7816-469647

Started

Container started

opendatahub

kubelet

test-curl-7816-469647

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7816-469647

AddedInterface

Add eth0 [10.133.0.167/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Unhealthy

Readiness probe failed: Get "http://10.133.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

ProbeError

Readiness probe error: Get "http://10.133.0.160:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

opendatahub

multus

test-curl-7816-f108c6

AddedInterface

Add eth0 [10.133.0.168/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7816-f108c6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Created

Container created

opendatahub

kubelet

test-curl-7816-469647

Killing

Stopping container test-curl-7816-469647

opendatahub

kubelet

test-curl-7816-f108c6

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7816-f108c6

Created

Container created

opendatahub

kubelet

test-curl-7816-f108c6

Killing

Stopping container test-curl-7816-f108c6

opendatahub

multus

test-curl-7816-4986cc

AddedInterface

Add eth0 [10.133.0.169/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7816-4986cc

Created

Container created

opendatahub

kubelet

test-curl-7816-4986cc

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7816-4986cc

Started

Container started

opendatahub

kubelet

test-curl-7816-f861bc

Started

Container started

opendatahub

kubelet

test-curl-7816-f861bc

Created

Container created

opendatahub

kubelet

test-curl-7816-f861bc

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7816-f861bc

AddedInterface

Add eth0 [10.133.0.170/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7816-4986cc

Killing

Stopping container test-curl-7816-4986cc
(x5)

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

ProbeError

Startup probe error: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ait-b3690233-d5d6b4bbc-8227h

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Unhealthy

Startup probe failed: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b3690233-6f6c5cd679-7cc8f

Killing

Stopping container payload-pre-processing

opendatahub

kubelet

test-curl-7816-f861bc

Killing

Stopping container test-curl-7816-f861bc
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-b3690233-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-b3690233-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-autores-147981

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-147981

openshift-ingress

replicaset-controller

e2e-autores-147981-openshift-default-76d9954467

SuccessfulCreate

Created pod: e2e-autores-147981-openshift-default-76d9954467-94hjz

openshift-ingress

deployment-controller

e2e-autores-147981-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-147981-openshift-default-76d9954467 from 0 to 1

openshift-ingress

multus

e2e-autores-147981-openshift-default-76d9954467-94hjz

AddedInterface

Add eth0 [10.133.0.171/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-derive-3261bdf2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-3261bdf2-openshift-default-649b9ff6d9 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-derive-3261bdf2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-3261bdf2

openshift-ingress

replicaset-controller

e2e-derive-3261bdf2-openshift-default-649b9ff6d9

SuccessfulCreate

Created pod: e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

openshift-ingress

multus

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

AddedInterface

Add eth0 [10.133.0.172/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-b3690233-openshift-default-7bcfb888c7-tcb4r

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-autores-147981

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-147981-77f5d75b89 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-147981-7c5d5b86ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8

openshift-ingress

multus

payload-processing-e2e-autores-147981-77f5d75b89-sg26f

AddedInterface

Add eth0 [10.132.0.104/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-147981-7c5d5b86ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-147981-7c5d5b86ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-147981: serviceaccount "payload-processing-e2e-autores-147981" not found

openshift-ingress

multus

payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8

AddedInterface

Add eth0 [10.132.0.103/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-147981-77f5d75b89

SuccessfulCreate

Created pod: payload-processing-e2e-autores-147981-77f5d75b89-sg26f

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-147981

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-147981-7c5d5b86ff from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-147981-77f5d75b89

FailedCreate

Error creating: pods "payload-processing-e2e-autores-147981-77f5d75b89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-147981: serviceaccount "payload-processing-e2e-autores-147981" not found

openshift-ingress

kubelet

payload-processing-e2e-autores-147981-77f5d75b89-sg26f

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-autores-147981-77f5d75b89-sg26f

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-autores-147981-77f5d75b89-sg26f

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-3261bdf2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-3261bdf2-779c6576f9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-3261bdf2-657dd64c9f

SuccessfulCreate

Created pod: payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-3261bdf2-657dd64c9f

FailedCreate

Error creating: pods "payload-processing-e2e-derive-3261bdf2-657dd64c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-3261bdf2: serviceaccount "payload-processing-e2e-derive-3261bdf2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-derive-3261bdf2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-3261bdf2-657dd64c9f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-3261bdf2: serviceaccount "payload-processing-e2e-derive-3261bdf2" not found

openshift-ingress

multus

payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h

AddedInterface

Add eth0 [10.132.0.106/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs

AddedInterface

Add eth0 [10.132.0.105/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-autores-147981-openshift-default-76d9954467-94hjz

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-autores-147981-openshift-default-76d9954467-94hjz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-autores-147981-openshift-default-76d9954467-94hjz

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-147981-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-147981-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-3261bdf2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-3261bdf2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

ProbeError

Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

Unhealthy

Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-autores-147981-openshift-default-76d9954467-94hjz

ProbeError

Startup probe error: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-autores-147981-openshift-default-76d9954467-94hjz

Unhealthy

Startup probe failed: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-3261bdf2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-3261bdf2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-derive-3261bdf2-657dd64c9f-w8h2h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-3261bdf2-779c6576f9-dm4bs

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-147981-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-147981-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-worker-w1-f72bc6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-f72bc6

openshift-ingress

deployment-controller

e2e-worker-w1-f72bc6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-f72bc6-openshift-default-569b7cb555 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-f72bc6-openshift-default-569b7cb555

SuccessfulCreate

Created pod: e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

openshift-ingress

multus

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

AddedInterface

Add eth0 [10.133.0.177/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-derive-3261bdf2-openshift-default-649b9ff6d9-vk2q9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-f72bc6: serviceaccount "payload-processing-e2e-worker-w1-f72bc6" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-f72bc6: serviceaccount "payload-processing-e2e-worker-w1-f72bc6" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w1-f72bc6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w1-f72bc6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445

AddedInterface

Add eth0 [10.134.0.129/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6

AddedInterface

Add eth0 [10.132.0.108/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-147981-77f5d75b89-sg26f

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f

SuccessfulDelete

Deleted pod: e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-g2wqq

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-extmod-b-2bb4ef-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f from 2 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-147981-7c5d5b86ff-d4mf8

Killing

Stopping container payload-pre-processing

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-2bb4ef-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 4 to 3

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 6 to 5

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-znwrc

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-fvpd2

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 6 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-7q22p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-b5t2t

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-autores-147981-openshift-default-76d9954467-94hjz

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

AddedInterface

Add eth0 [10.132.0.109/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-explicit-5a4525-openshift-default-cd8fbcb69

SuccessfulCreate

Created pod: e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

openshift-ingress

deployment-controller

e2e-explicit-5a4525-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-5a4525-openshift-default-cd8fbcb69 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-explicit-5a4525

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-5a4525

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lj8bq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-lk65s

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 5 to 3

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-5a4525-56bf585947

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-5a4525-56bf585947

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-5a4525-56bf585947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-5a4525: serviceaccount "payload-processing-e2e-explicit-5a4525" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-5a4525-85499b7f45

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-5a4525-85499b7f45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-5a4525: serviceaccount "payload-processing-e2e-explicit-5a4525" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-5a4525

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-5a4525-56bf585947 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-5a4525

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-5a4525-85499b7f45 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl

AddedInterface

Add eth0 [10.132.0.111/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-5a4525-85499b7f45

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf

openshift-ingress

kubelet

payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf

AddedInterface

Add eth0 [10.132.0.112/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

ProbeError

Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-9dc2m

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

Unhealthy

Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 5 to 4

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-shared-b-w4-623b4b-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47 from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-623b4b-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47

SuccessfulDelete

Deleted pod: e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-xkxn8

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-6z2t7

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-f72bc6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-f72bc6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

ProbeError

Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

Unhealthy

Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-f72bc6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-f72bc6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-api-a-d0c9b9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d0c9b9

openshift-ingress

deployment-controller

e2e-api-a-d0c9b9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-d0c9b9-openshift-default-ffc94f6c9 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9

SuccessfulCreate

Created pod: e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

openshift-ingress

multus

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

AddedInterface

Add eth0 [10.133.0.196/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-d0c9b9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-d0c9b9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d0c9b9: serviceaccount "payload-processing-e2e-api-a-d0c9b9" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d0c9b9: serviceaccount "payload-processing-e2e-api-a-d0c9b9" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

openshift-ingress

kubelet

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

AddedInterface

Add eth0 [10.132.0.115/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

openshift-ingress

multus

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

AddedInterface

Add eth0 [10.134.0.133/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

Unhealthy

Readiness probe failed: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

ProbeError

Readiness probe error: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-5a4525-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-5a4525-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w1-f72bc6-openshift-default-569b7cb555

SuccessfulCreate

Created pod: e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-f72bc6-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w1-f72bc6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w1-f72bc6-openshift-default-569b7cb555 from 1 to 2

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

Started

Container started

openshift-ingress

multus

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

AddedInterface

Add eth0 [10.134.0.134/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

Created

Container created

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

Unhealthy

Startup probe failed: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

ProbeError

Startup probe error: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-d0c9b9-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-d0c9b9-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-tf5fr

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target
(x6)

openshift-ingress

kubelet

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

Unhealthy

Startup probe failed: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

ProbeError

Startup probe error: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-5a4525-56bf585947-mzjsl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-5a4525-85499b7f45-5mfwf

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-api-b-928a7a-openshift-default-6d557d6b67

SuccessfulCreate

Created pod: e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

openshift-ingress

deployment-controller

e2e-api-b-928a7a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-928a7a-openshift-default-6d557d6b67 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-b-928a7a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-928a7a

openshift-ingress

multus

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

AddedInterface

Add eth0 [10.133.0.211/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-928a7a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-928a7a-784d585f5 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-928a7a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-928a7a-79d54fb94f from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-928a7a-79d54fb94f

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-928a7a-79d54fb94f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-928a7a: serviceaccount "payload-processing-e2e-api-b-928a7a" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-928a7a-784d585f5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-928a7a-784d585f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-928a7a: serviceaccount "payload-processing-e2e-api-b-928a7a" not found

openshift-ingress

multus

payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl

AddedInterface

Add eth0 [10.132.0.123/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-928a7a-784d585f5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z

openshift-ingress

kubelet

payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-928a7a-79d54fb94f

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl

openshift-ingress

multus

payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z

AddedInterface

Add eth0 [10.132.0.122/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl

Started

Container started

openshift-ingress

replicaset-controller

e2e-mt-778b2bce-openshift-default-7685bc8b9c

SuccessfulCreate

Created pod: e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

openshift-ingress

multus

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

AddedInterface

Add eth0 [10.133.0.212/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-mt-778b2bce-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-778b2bce-openshift-default-7685bc8b9c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-778b2bce

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-778b2bce

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-778b2bce

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-778b2bce-5645b6b76c from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-778b2bce

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-778b2bce-7f6cdb88c9 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

AddedInterface

Add eth0 [10.132.0.124/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-778b2bce-7f6cdb88c9

FailedCreate

Error creating: pods "payload-processing-e2e-mt-778b2bce-7f6cdb88c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-778b2bce: serviceaccount "payload-processing-e2e-mt-778b2bce" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

openshift-ingress

multus

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

AddedInterface

Add eth0 [10.132.0.125/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-778b2bce: serviceaccount "payload-processing-e2e-mt-778b2bce" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-778b2bce-7f6cdb88c9

SuccessfulCreate

Created pod: payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

Started

Container started

openshift-ingress

kubelet

e2e-explicit-5a4525-openshift-default-cd8fbcb69-t6tv4

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-noten-53f37e-openshift-default-6766bfb7f

SuccessfulCreate

Created pod: e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-d0c9b9-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-d0c9b9-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-noten-53f37e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-53f37e

openshift-ingress

deployment-controller

e2e-noten-53f37e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-53f37e-openshift-default-6766bfb7f from 0 to 1

openshift-ingress

kubelet

e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl

Created

Container created

openshift-ingress

multus

e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl

AddedInterface

Add eth0 [10.133.0.213/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-53f37e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-53f37e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-noten-53f37e-openshift-default-6766bfb7f-qrkfl

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

Unhealthy

Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

ProbeError

Readiness probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Unhealthy

Readiness probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

ProbeError

Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-e07583

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-e07583

openshift-ingress

replicaset-controller

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65

SuccessfulCreate

Created pod: e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

openshift-ingress

deployment-controller

e2e-shared-a-w3-e07583-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-e07583-openshift-default-6554fb6f65 from 0 to 1

openshift-ingress

multus

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

AddedInterface

Add eth0 [10.133.0.217/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-e07583: serviceaccount "payload-processing-e2e-shared-a-w3-e07583" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-e07583

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-e07583

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-e07583-7749bbd66c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-e07583: serviceaccount "payload-processing-e2e-shared-a-w3-e07583" not found

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

AddedInterface

Add eth0 [10.132.0.128/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd

AddedInterface

Add eth0 [10.134.0.138/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-928a7a-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd

Created

Container created

openshift-ingress

deployment-controller

e2e-api-b-928a7a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-928a7a-openshift-default-6d557d6b67 from 1 to 2

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-api-b-928a7a-openshift-default-6d557d6b67

SuccessfulCreate

Created pod: e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

openshift-ingress

multus

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

AddedInterface

Add eth0 [10.134.0.139/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

Created

Container created

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

Unhealthy

Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

ProbeError

Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

ProbeError

Startup probe error: Get "http://10.133.0.211:15021/healthz/ready": dial tcp 10.133.0.211:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

Unhealthy

Startup probe failed: Get "http://10.133.0.211:15021/healthz/ready": dial tcp 10.133.0.211:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-928a7a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-928a7a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

Killing

Stopping container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

Unhealthy

Startup probe failed: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

ProbeError

Startup probe error: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-api-a-d0c9b9-5fffc7bfd9-bdzlx

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.132.0.115:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d0c9b9-5d6754f895-7mp4z

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.133:50052" within 1s: context deadline exceeded

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-928a7a-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-928a7a-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-api-a-d0c9b9-openshift-default-ffc94f6c9-4l9hn

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-778b2bce-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-778b2bce-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-api-b-928a7a-79d54fb94f-7dlcl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-928a7a-784d585f5-cg49z

Killing

Stopping container payload-pre-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-778b2bce-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-778b2bce-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-5ba53f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-5ba53f

openshift-ingress

deployment-controller

e2e-shared-b-w3-5ba53f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc

SuccessfulCreate

Created pod: e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

Created

Container created

openshift-ingress

multus

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

AddedInterface

Add eth0 [10.132.0.131/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-e07583-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-e07583-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-mt-90a96161

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-90a96161
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

ProbeError

Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-5ba53f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

openshift-ingress

deployment-controller

e2e-mt-90a96161-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-90a96161-openshift-default-85fd94cfcc from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-5ba53f: serviceaccount "payload-processing-e2e-shared-b-w3-5ba53f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-5ba53f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-5ba53f-85689465fc from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-5ba53f: serviceaccount "payload-processing-e2e-shared-b-w3-5ba53f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

openshift-ingress

replicaset-controller

e2e-mt-90a96161-openshift-default-85fd94cfcc

SuccessfulCreate

Created pod: e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

Unhealthy

Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

AddedInterface

Add eth0 [10.132.0.132/23] from ovn-kubernetes

openshift-ingress

multus

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

AddedInterface

Add eth0 [10.133.0.220/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

AddedInterface

Add eth0 [10.134.0.140/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-90a96161-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-90a96161-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-mhgpb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-b-928a7a-openshift-default-6d557d6b67-d8hw8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq

AddedInterface

Add eth0 [10.132.0.134/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-90a96161-86597f9cbc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-90a96161-694f45b47c

SuccessfulCreate

Created pod: payload-processing-e2e-mt-90a96161-694f45b47c-t985m
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-90a96161-86597f9cbc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-90a96161-86597f9cbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-90a96161: serviceaccount "payload-processing-e2e-mt-90a96161" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-90a96161

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-90a96161-694f45b47c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-90a96161

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-90a96161-86597f9cbc from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-90a96161-694f45b47c

FailedCreate

Error creating: pods "payload-processing-e2e-mt-90a96161-694f45b47c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-90a96161: serviceaccount "payload-processing-e2e-mt-90a96161" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-90a96161-694f45b47c-t985m

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq

Created

Container created

openshift-ingress

multus

payload-processing-e2e-mt-90a96161-694f45b47c-t985m

AddedInterface

Add eth0 [10.132.0.135/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-90a96161-694f45b47c-t985m

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-90a96161-694f45b47c-t985m

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

deployment-controller

e2e-worker-w0-555d1e-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled down replica set e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87 from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-555d1e-openshift-default

SuccessfulRescale

(combined from similar events): New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87

SuccessfulDelete

Deleted pod: e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-25r84

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 3 to 2

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-kk6xh

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-e07583-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-e07583-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

multus

test-curl-7807-0ab152

AddedInterface

Add eth0 [10.132.0.136/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7807-0ab152

Created

Container created

opendatahub

kubelet

test-curl-7807-0ab152

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7807-0ab152

Started

Container started

opendatahub

kubelet

test-curl-7807-0ab152

Killing

Stopping container test-curl-7807-0ab152

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

Unhealthy

Readiness probe failed: Get "http://10.132.0.131:15021/healthz/ready": read tcp 10.132.0.2:33880->10.132.0.131:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

ProbeError

Readiness probe error: Get "http://10.132.0.131:15021/healthz/ready": read tcp 10.132.0.2:33880->10.132.0.131:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

ProbeError

Startup probe error: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

Unhealthy

Startup probe failed: Get "http://10.132.0.131:15021/healthz/ready": dial tcp 10.132.0.131:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f

SuccessfulDelete

Deleted pod: e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-4be645-openshift-default

SuccessfulRescale

(combined from similar events): New size: 1; reason: All metrics below target
(x2)

openshift-ingress

deployment-controller

e2e-worker-w2-4be645-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled down replica set e2e-worker-w2-4be645-openshift-default-ccc9d9d8f from 2 to 1

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-47p6q

Killing

Stopping container istio-proxy
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-5ba53f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-5ba53f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

Started

Container started
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-mkwdq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-29vx2

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x5)

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

Unhealthy

Startup probe failed: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

ProbeError

Startup probe error: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-5ba53f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-5ba53f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-90a96161-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-90a96161-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-extmod-b-12b8da-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-12b8da-openshift-default-7df679ddc7 from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-12b8da-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-extmod-b-12b8da-openshift-default-7df679ddc7

SuccessfulDelete

Deleted pod: e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-wg526

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-cdz82

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-778b2bce-7f6cdb88c9

SuccessfulCreate

Created pod: payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-778b2bce

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-778b2bce-5645b6b76c from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-778b2bce-7f6cdb88c9

FailedCreate

Error creating: pods "payload-processing-e2e-mt-778b2bce-7f6cdb88c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-778b2bce: serviceaccount "payload-processing-e2e-mt-778b2bce" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-778b2bce: serviceaccount "payload-processing-e2e-mt-778b2bce" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-778b2bce

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-778b2bce-7f6cdb88c9 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh

AddedInterface

Add eth0 [10.132.0.141/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw

AddedInterface

Add eth0 [10.132.0.140/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w1-f72bc6-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w1-f72bc6-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w1-f72bc6-openshift-default-569b7cb555 from 2 to 1

openshift-ingress

replicaset-controller

e2e-worker-w1-f72bc6-openshift-default-569b7cb555

SuccessfulDelete

Deleted pod: e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-kxt9m

Killing

Stopping container istio-proxy

opendatahub

multus

test-curl-7807-4143b6

AddedInterface

Add eth0 [10.133.0.227/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7807-4143b6

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7807-4143b6

Started

Container started

opendatahub

kubelet

test-curl-7807-4143b6

Created

Container created

opendatahub

kubelet

test-curl-7807-4143b6

Killing

Stopping container test-curl-7807-4143b6

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-90a96161-86597f9cbc-vc9cq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-90a96161-694f45b47c-t985m

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-9ba664-f5986c96c-4rnrd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-9ba664-54cdd94959-2r722

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-mt-778b2bce-7f6cdb88c9-vbslh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-778b2bce-5645b6b76c-595vw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-9ba664-openshift-default-5b57547dc7-sgdhx

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-778b2bce-openshift-default-7685bc8b9c-h76xn

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-97tgx

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-55fbd65575 from 10 to 6

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-cknl5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-97tgx

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-cknl5

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-pxflc

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55fbd65575-jbt2l

openshift-ingress

kubelet

e2e-extmod-b-12b8da-openshift-default-7df679ddc7-n47dh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-jbt2l

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-pxflc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

Unhealthy

Readiness probe failed: Get "http://10.133.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-mt-90a96161-openshift-default-85fd94cfcc-gblmd

ProbeError

Readiness probe error: Get "http://10.133.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-12b8da-7496c4f78f-2xs4l

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-12b8da-84bc4c4dc5-2lwjw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-4be645-58d4569f87-754xp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-4be645-544cb69f75-b77xx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-2m48q
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 6 to 10
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-t9gjp

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-q8r79

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55fbd65575

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55fbd65575-4q677

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-q8r79

AddedInterface

Add eth0 [10.133.0.238/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4q677

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4q677

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-t9gjp

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-4q677

AddedInterface

Add eth0 [10.134.0.144/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-q8r79

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-q8r79

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-q8r79

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-2m48q

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4q677

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-t9gjp

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-t9gjp

AddedInterface

Add eth0 [10.133.0.236/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-t9gjp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55fbd65575-2m48q

AddedInterface

Add eth0 [10.133.0.237/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-2m48q

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-2m48q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4q677

ProbeError

Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4q677

Unhealthy

Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-t9gjp

Unhealthy

Startup probe failed: Get "http://10.133.0.236:15021/healthz/ready": dial tcp 10.133.0.236:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-q8r79

ProbeError

Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-2m48q

ProbeError

Startup probe error: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-t9gjp

ProbeError

Startup probe error: Get "http://10.133.0.236:15021/healthz/ready": dial tcp 10.133.0.236:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-2m48q

Unhealthy

Startup probe failed: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-q8r79

Unhealthy

Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w2-4be645-openshift-default-ccc9d9d8f-nfjnl

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

ProbeError

Readiness probe error: Get "http://10.132.0.75:8080/ready": dial tcp 10.132.0.75:8080: connect: connection refused body:

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

Unhealthy

Readiness probe failed: Get "http://10.132.0.75:8080/ready": dial tcp 10.132.0.75:8080: connect: connection refused

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-rdlmf

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-rdlmf

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

e2e-worker-w4-c0c28f-openshift-default-78946b8b89

SuccessfulDelete

Deleted pod: e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-sbv6m

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-c0c28f-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w4-c0c28f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-c0c28f-openshift-default-78946b8b89 from 2 to 1

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulDelete

Deleted pod: e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

Unhealthy

Readiness probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9hrdc

ProbeError

Readiness probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t

Killing

Stopping container payload-processing

opendatahub

kubelet

test-curl-7813-09c805

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7813-09c805

Created

Container created

opendatahub

kubelet

test-curl-7813-09c805

Started

Container started

opendatahub

multus

test-curl-7813-09c805

AddedInterface

Add eth0 [10.132.0.142/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7813-b1ec9c

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7813-b1ec9c

AddedInterface

Add eth0 [10.132.0.143/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7813-09c805

Killing

Stopping container test-curl-7813-09c805

opendatahub

kubelet

test-curl-7813-b1ec9c

Created

Container created

opendatahub

kubelet

test-curl-7813-b1ec9c

Started

Container started

opendatahub

kubelet

test-curl-7813-b1ec9c

Killing

Stopping container test-curl-7813-b1ec9c
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-f1e1c2-6db8c44df7-2788t

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-f1e1c2-bf5f7dfff-zdmfr

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-a-w4-f1e1c2-openshift-default-5b9dc59c9f-7cjbn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-555d1e-758c9888f-6hs9g

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-555d1e-6d9867c5f9-drnpk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-623b4b-5dcc7b576f-w768r

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-623b4b-774f7fb84c-kzrt5

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-b-w4-623b4b-openshift-default-5d6db7b47-2dt46

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-555d1e-openshift-default-7dd9fbcb87-wfqsm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-c0c28f-68fd74bd7f-88snm

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-c0c28f-7854685bb7-dqdp5

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w4-c0c28f-openshift-default-78946b8b89-97hfj

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 1 to 3

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

Started

Container started

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

AddedInterface

Add eth0 [10.134.0.145/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

AddedInterface

Add eth0 [10.132.0.146/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

Started

Container started
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

ProbeError

Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

ProbeError

Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

Unhealthy

Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

Unhealthy

Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-e07583-openshift-default-6554fb6f65-rgcfb

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-e07583-65b978697f-hjgn5

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-e07583-7749bbd66c-mkjfd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-5ba53f-85689465fc-jq784

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-5ba53f-77b6455cc9-6kd9z

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-b-w3-5ba53f-openshift-default-6d979b9bdc-slv8l

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-a-944506-openshift-default-855597c485-fdk9x

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-673ba8-6ffb686999-d8kvk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-673ba8-7664594f7f-zrsqb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-b-673ba8-openshift-default-755484574f-x9zcn

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-944506-5bcfb5fb67-8cq9w

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-944506-786f89d5dc-tmm5j

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w1-f72bc6-74b59bb48c-j6445

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w1-f72bc6-9b78d7d7c-2mbq6

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-866159-7ff65f964c-jz6w9

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-866159-7b94c5b8c9-n76dx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-866159-openshift-default

SuccessfulRescale

(combined from similar events): New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

AddedInterface

Add eth0 [10.132.0.147/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-866159-openshift-default-856f7748b9

SuccessfulCreate

Created pod: e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

Created

Container created
(x4)

openshift-ingress

deployment-controller

e2e-worker-w3-866159-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set e2e-worker-w3-866159-openshift-default-856f7748b9 from 3 to 4

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

Unhealthy

Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

ProbeError

Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-9rrgf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-28fbz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-j9vwr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-866159-openshift-default-856f7748b9-5r78m

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w1-f72bc6-openshift-default-569b7cb555-gvp4p

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-a-bf0b0d-openshift-default-697867ff97-84mxd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-bf0b0d-84686fb7dc-wccfl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-bf0b0d-58bbc75bb7-zp2nm

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-2bb4ef-openshift-default-5bf7b7c64f-2vhb4

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-2bb4ef-c686dbc8c-xp6sk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-2bb4ef-b4ff578f9-lhcd6

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

opendatahub

kubelet

maas-controller-5db8dc9fd8-76ms5

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-5db8dc9fd8

SuccessfulDelete

Deleted pod: maas-controller-5db8dc9fd8-76ms5

opendatahub

replicaset-controller

maas-controller-5db8dc9fd8

SuccessfulCreate

Created pod: maas-controller-5db8dc9fd8-jdk62

opendatahub

multus

maas-controller-5db8dc9fd8-jdk62

AddedInterface

Add eth0 [10.134.0.146/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-5db8dc9fd8-jdk62

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8"

opendatahub

kubelet

maas-controller-5db8dc9fd8-jdk62

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8" in 1.791s (1.791s including waiting). Image size: 200094742 bytes.

opendatahub

kubelet

maas-controller-5db8dc9fd8-jdk62

Started

Container started

opendatahub

kubelet

maas-controller-5db8dc9fd8-jdk62

Created

Container created

opendatahub

maas-controller-5db8dc9fd8-jdk62_9094828a-2769-4469-ace5-b6cb3bc1bef3

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-5db8dc9fd8-jdk62_9094828a-2769-4469-ace5-b6cb3bc1bef3 became leader
(x2)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-5db8dc9fd8 from 1 to 0

opendatahub

kubelet

maas-controller-5db8dc9fd8-jdk62

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-5db8dc9fd8

SuccessfulDelete

Deleted pod: maas-controller-5db8dc9fd8-jdk62
(x3)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-5db8dc9fd8 from 0 to 1

opendatahub

replicaset-controller

maas-controller-5db8dc9fd8

SuccessfulCreate

Created pod: maas-controller-5db8dc9fd8-zlxn4

opendatahub

kubelet

maas-controller-5db8dc9fd8-zlxn4

Started

Container started

opendatahub

kubelet

maas-controller-5db8dc9fd8-zlxn4

Created

Container created

opendatahub

kubelet

maas-controller-5db8dc9fd8-zlxn4

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8" in 422ms (422ms including waiting). Image size: 200094742 bytes.

opendatahub

multus

maas-controller-5db8dc9fd8-zlxn4

AddedInterface

Add eth0 [10.134.0.148/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-5db8dc9fd8-zlxn4

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:04290f7dfdd04a75381308df807f81f2f0048b4928d28649bf3cb40f8d6f4dd8"

opendatahub

maas-controller-5db8dc9fd8-zlxn4_8f0681c7-aca0-4e71-9162-99fe6ac135f1

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-5db8dc9fd8-zlxn4_8f0681c7-aca0-4e71-9162-99fe6ac135f1 became leader

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-678d87766f-4flwn

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-4flwn

Killing

Stopping container manager

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-678d87766f from 1 to 0

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

NeedsReinstall

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-fgnlf

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

ComponentUnhealthy

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x3)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-fgnlf

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-fgnlf

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-fgnlf

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-fgnlf

AddedInterface

Add eth0 [10.134.0.150/23] from ovn-kubernetes
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-fgnlf_bb744c8e-6459-478d-8318-726a04f4ed29

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-fgnlf_bb744c8e-6459-478d-8318-726a04f4ed29 became leader

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-q8r79

Unhealthy

Readiness probe failed: Get "http://10.133.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-2m48q

ProbeError

Readiness probe error: Get "http://10.133.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-2m48q

Unhealthy

Readiness probe failed: Get "http://10.133.0.237:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-q8r79

ProbeError

Readiness probe error: Get "http://10.133.0.238:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-ait-87f75f6f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-87f75f6f

openshift-ingress

deployment-controller

e2e-ait-87f75f6f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-87f75f6f-openshift-default-5fc6745d5f from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-87f75f6f-openshift-default-5fc6745d5f

SuccessfulCreate

Created pod: e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

openshift-ingress

multus

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

AddedInterface

Add eth0 [10.134.0.164/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-87f75f6f-67d67b585c

FailedCreate

Error creating: pods "payload-processing-e2e-ait-87f75f6f-67d67b585c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-87f75f6f: serviceaccount "payload-processing-e2e-ait-87f75f6f" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-87f75f6f-55db889895

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-87f75f6f-55db889895-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-87f75f6f: serviceaccount "payload-processing-e2e-ait-87f75f6f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-87f75f6f-55db889895

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7

openshift-ingress

deployment-controller

payload-processing-e2e-ait-87f75f6f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-87f75f6f-67d67b585c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-87f75f6f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-87f75f6f-55db889895 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-87f75f6f-67d67b585c

SuccessfulCreate

Created pod: payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk

openshift-ingress

multus

payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk

AddedInterface

Add eth0 [10.132.0.150/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7

AddedInterface

Add eth0 [10.132.0.149/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Unhealthy

Readiness probe failed: Get "http://10.134.0.164:15021/healthz/ready": read tcp 10.134.0.2:35926->10.134.0.164:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

ProbeError

Readiness probe error: Get "http://10.134.0.164:15021/healthz/ready": read tcp 10.134.0.2:35926->10.134.0.164:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Unhealthy

Startup probe failed: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

ProbeError

Startup probe error: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-87f75f6f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-87f75f6f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-ait-87f75f6f-67d67b585c-k86hk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-87f75f6f-55db889895-7hpp7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-87f75f6f-openshift-default-5fc6745d5f-jjvcc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.Service llm/e2e-prio-pool-170c8925-epp-service

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.Deployment llm/e2e-prio-pool-170c8925-kserve-router-scheduler

llm

kubelet

e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb

Started

Container started

llm

kubelet

e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

multus

e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb

AddedInterface

Add eth0 [10.132.0.151/23] from ovn-kubernetes

llm

kubelet

e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb

Created

Container created

llm

deployment-controller

e2e-prio-pool-170c8925-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc from 0 to 1

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.Deployment llm/e2e-prio-pool-170c8925-kserve

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.ServiceAccount llm/e2e-prio-pool-170c8925-epp-sa

llm

replicaset-controller

e2e-prio-pool-170c8925-kserve-7959dfcd4

SuccessfulCreate

Created pod: e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.RoleBinding llm/e2e-prio-pool-170c8925-epp-rb

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.Role llm/e2e-prio-pool-170c8925-epp-role

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.ClusterRoleBinding /llm-e2e-prio-pool-170c8925-epp-auth-rb

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.Service llm/e2e-prio-pool-170c8925-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

Created v1.Secret llm/e2e-prio-pool-170c8925-kserve-self-signed-certs

llm

deployment-controller

e2e-prio-pool-170c8925-kserve

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-170c8925-kserve-7959dfcd4 from 0 to 1

llm

replicaset-controller

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc

SuccessfulCreate

Created pod: e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

llm

kubelet

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

Pulling

Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827"

llm

multus

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

AddedInterface

Add eth0 [10.134.0.170/23] from ovn-kubernetes
(x9)

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Created

(combined from similar events): Created v1.DestinationRule llm/e2e-prio-pool-170c8925-kserve-shadow-svc

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Updated

Updated v1.HTTPRoute llm/e2e-prio-pool-170c8925-kserve-route

llm

kubelet

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

Started

Container started

llm

kubelet

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

Pulled

Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827" in 1.976s (1.976s including waiting). Image size: 184206420 bytes.

llm

kubelet

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

Created

Container created
(x2)

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Updated

Updated v1.Secret llm/e2e-prio-pool-170c8925-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

LLMInferenceServiceReady

LLMInferenceService [e2e-prio-pool-170c8925] is Ready

llm

LLMInferenceServiceController

e2e-prio-pool-170c8925

Deleted

Deleted v1.ClusterRoleBinding /llm-e2e-prio-pool-170c8925-epp-auth-rb

llm

kubelet

e2e-prio-pool-170c8925-kserve-7959dfcd4-2dmcb

Killing

Stopping container main

llm

kubelet

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

Killing

Stopping container main

llm

replicaset-controller

e2e-same-id-a-2e2eed0e-kserve-59d59f66bf

SuccessfulCreate

Created pod: e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.Service llm/e2e-same-id-a-2e2eed0e-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.DestinationRule llm/e2e-same-id-a-2e2eed0e-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.PodMonitor llm/e2e-same-id-a-2e2eed0e-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.HTTPRoute llm/e2e-same-id-a-2e2eed0e-kserve-route

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.PodMonitor llm/e2e-same-id-a-2e2eed0e-kserve-llmisvc-engine-default

llm

deployment-controller

e2e-same-id-a-2e2eed0e-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-a-2e2eed0e-kserve-59d59f66bf from 0 to 1

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.Secret llm/e2e-same-id-a-2e2eed0e-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.Deployment llm/e2e-same-id-a-2e2eed0e-kserve

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Created

Created v1.NetworkPolicy llm/e2e-same-id-a-2e2eed0e-prometheus-scraping

llm

kubelet

e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z

Started

Container started

llm

multus

e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z

AddedInterface

Add eth0 [10.132.0.152/23] from ovn-kubernetes

llm

kubelet

e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z

Created

Container created
(x2)

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

Updated

Updated v1.Secret llm/e2e-same-id-a-2e2eed0e-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-a-2e2eed0e] is Ready

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.PodMonitor llm/e2e-same-id-b-2e2eed0e-kserve-llmisvc-engine

llm

kubelet

e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq

Started

Container started

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.NetworkPolicy llm/e2e-same-id-b-2e2eed0e-prometheus-scraping

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.Service llm/e2e-same-id-b-2e2eed0e-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.Deployment llm/e2e-same-id-b-2e2eed0e-kserve

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.DestinationRule llm/e2e-same-id-b-2e2eed0e-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.Secret llm/e2e-same-id-b-2e2eed0e-kserve-self-signed-certs

llm

deployment-controller

e2e-same-id-b-2e2eed0e-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-b-2e2eed0e-kserve-7f9969b75c from 0 to 1

llm

kubelet

e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq

Created

Container created

llm

kubelet

e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.HTTPRoute llm/e2e-same-id-b-2e2eed0e-kserve-route

llm

replicaset-controller

e2e-same-id-b-2e2eed0e-kserve-7f9969b75c

SuccessfulCreate

Created pod: e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Created

Created v1.PodMonitor llm/e2e-same-id-b-2e2eed0e-kserve-llmisvc-engine-default

llm

multus

e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq

AddedInterface

Add eth0 [10.132.0.153/23] from ovn-kubernetes
(x5)

llm

kubelet

e2e-prio-pool-170c8925-kserve-router-scheduler-569997cfbc-vlx2q

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

Updated

Updated v1.Secret llm/e2e-same-id-b-2e2eed0e-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-b-2e2eed0e] is Ready
(x8)

llm

LLMInferenceServiceController

e2e-same-id-b-2e2eed0e

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-a-2e2eed0e in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-b-2e2eed0e; safe to ignore if traffic splitting is planned
(x2)

llm

LLMInferenceServiceController

e2e-same-id-a-2e2eed0e

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-b-2e2eed0e in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-a-2e2eed0e; safe to ignore if traffic splitting is planned

llm

kubelet

e2e-same-id-a-2e2eed0e-kserve-59d59f66bf-h254z

Killing

Stopping container main

llm

kubelet

e2e-same-id-b-2e2eed0e-kserve-7f9969b75c-22wpq

Killing

Stopping container main
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-9pqlk

Unhealthy

Readiness probe failed: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-9pqlk

ProbeError

Readiness probe error: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-ipp-a-b21aae-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-b21aae-openshift-default-7f5b5c687f from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f

SuccessfulCreate

Created pod: e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

openshift-ingress

multus

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

AddedInterface

Add eth0 [10.134.0.174/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-b21aae

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-b21aae
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-b21aae-748f845497

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-b21aae-748f845497-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b21aae: serviceaccount "payload-processing-e2e-ipp-a-b21aae" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-b21aae

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-b21aae-748f845497 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b21aae: serviceaccount "payload-processing-e2e-ipp-a-b21aae" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-b21aae

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-b21aae-748f845497

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95

Created

Container created

openshift-ingress

multus

payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg

AddedInterface

Add eth0 [10.132.0.156/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95

AddedInterface

Add eth0 [10.132.0.155/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-6c6bc46cdc-f4v95

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

Unhealthy

Readiness probe failed: Get "http://10.134.0.174:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-748f845497-z4ztg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

ProbeError

Readiness probe error: Get "http://10.134.0.174:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

Created

Container created
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b21aae: serviceaccount "payload-processing-e2e-ipp-a-b21aae" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-b21aae

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-b21aae

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-b21aae-6bc4bd8db from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b21aae: serviceaccount "payload-processing-e2e-ipp-a-b21aae" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

AddedInterface

Add eth0 [10.134.0.175/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

AddedInterface

Add eth0 [10.134.0.176/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

ProbeError

Startup probe error: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

Unhealthy

Startup probe failed: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.604s (2.604s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.488s (2.488s including waiting). Image size: 166620835 bytes.

openshift-ingress

replicaset-controller

e2e-ipp-b-be6018-openshift-default-6959f6cfbf

SuccessfulCreate

Created pod: e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

openshift-ingress

deployment-controller

e2e-ipp-b-be6018-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-be6018-openshift-default-6959f6cfbf from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-be6018

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-be6018

openshift-ingress

multus

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

AddedInterface

Add eth0 [10.132.0.157/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6018: serviceaccount "payload-processing-e2e-ipp-b-be6018" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-be6018

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-be6018-58f558f76c from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-be6018

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-be6018-66d69866c from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-be6018-66d69866c

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-be6018-66d69866c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6018: serviceaccount "payload-processing-e2e-ipp-b-be6018" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-be6018-66d69866c

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq

openshift-ingress

multus

payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7

AddedInterface

Add eth0 [10.132.0.159/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq

AddedInterface

Add eth0 [10.132.0.158/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-b21aae-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-b21aae-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-58f558f76c-h5mbq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-66d69866c-vlww7

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-be6018

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-be6018-6775fc74f6 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-be6018

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-be6018-7f446979f from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-be6018-6775fc74f6

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-be6018-6775fc74f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6018: serviceaccount "payload-processing-e2e-ipp-b-be6018" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-be6018-7f446979f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-be6018-7f446979f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-be6018: serviceaccount "payload-processing-e2e-ipp-b-be6018" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-be6018-7f446979f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9
(x5)

openshift-ingress

kubelet

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

ProbeError

Startup probe error: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

Unhealthy

Startup probe failed: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9

AddedInterface

Add eth0 [10.132.0.160/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-be6018-6775fc74f6

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

AddedInterface

Add eth0 [10.132.0.161/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.88s (1.88s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.765s (1.765s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

Started

Container started
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-be6018-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-be6018-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-b21aae-5879fdf959-cms97

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-b21aae-6bc4bd8db-85wvr

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-b21aae-openshift-default-7f5b5c687f-b44jh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-be6018-7f446979f-x7gk9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-be6018-6775fc74f6-h829s

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-be6018-openshift-default-6959f6cfbf-clcpv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-574bdd7c8c-s7zkh

Created

Container created

openshift-ingress

multus

payload-pre-processing-574bdd7c8c-s7zkh

AddedInterface

Add eth0 [10.134.0.178/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-574bdd7c8c-s7zkh

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-574bdd7c8c-s7zkh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-574bdd7c8c

SuccessfulCreate

Created pod: payload-pre-processing-574bdd7c8c-s7zkh

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-574bdd7c8c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-7b98bb678c from 1 to 0

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-2btkh

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulDelete

Deleted pod: payload-pre-processing-7b98bb678c-2btkh

openshift-ingress

replicaset-controller

payload-processing-849d6b95cc

SuccessfulCreate

Created pod: payload-processing-849d6b95cc-pc2n9

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-849d6b95cc from 0 to 1

openshift-ingress

kubelet

payload-processing-849d6b95cc-pc2n9

Created

Container created

openshift-ingress

kubelet

payload-processing-849d6b95cc-pc2n9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-849d6b95cc-pc2n9

AddedInterface

Add eth0 [10.132.0.163/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-849d6b95cc-pc2n9

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulDelete

Deleted pod: payload-processing-55fbbbf9dc-rx5hk

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-55fbbbf9dc from 1 to 0

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-rx5hk

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-bbc4b5f5d from 0 to 1

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-8f7f4697c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-8f7f4697c

SuccessfulCreate

Created pod: payload-processing-8f7f4697c-89t4w

openshift-ingress

replicaset-controller

payload-pre-processing-bbc4b5f5d

SuccessfulCreate

Created pod: payload-pre-processing-bbc4b5f5d-l54ph

openshift-ingress

kubelet

payload-pre-processing-bbc4b5f5d-l54ph

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-bbc4b5f5d-l54ph

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-bbc4b5f5d-l54ph

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-bbc4b5f5d-l54ph

AddedInterface

Add eth0 [10.132.0.165/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-8f7f4697c-89t4w

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-8f7f4697c-89t4w

Started

Container started

openshift-ingress

kubelet

payload-processing-8f7f4697c-89t4w

Created

Container created

openshift-ingress

multus

payload-processing-8f7f4697c-89t4w

AddedInterface

Add eth0 [10.132.0.164/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-574bdd7c8c

SuccessfulDelete

Deleted pod: payload-pre-processing-574bdd7c8c-s7zkh

openshift-ingress

kubelet

payload-pre-processing-574bdd7c8c-s7zkh

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-849d6b95cc from 1 to 0

openshift-ingress

replicaset-controller

payload-processing-849d6b95cc

SuccessfulDelete

Deleted pod: payload-processing-849d6b95cc-pc2n9

openshift-ingress

kubelet

payload-processing-849d6b95cc-pc2n9

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-574bdd7c8c from 1 to 0

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5bcd7997 from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

SuccessfulCreate

Created pod: payload-processing-external-model-5bcd7997-n5n4n
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

FailedCreate

Error creating: pods "payload-processing-external-model-5bcd7997-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-n5n4n

Started

Container started

models-as-a-service

multus

payload-processing-external-model-5bcd7997-n5n4n

AddedInterface

Add eth0 [10.132.0.166/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-n5n4n

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-n5n4n

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-n5n4n

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-t9gjp

Unhealthy

Readiness probe failed: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-t9gjp

ProbeError

Readiness probe error: Get "http://10.133.0.236:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

ProbeError

Readiness probe error: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-8q5hv

Unhealthy

Readiness probe failed: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4q677

Unhealthy

Readiness probe failed: Get "http://10.134.0.144:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-4q677

ProbeError

Readiness probe error: Get "http://10.134.0.144:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-worker-single-main-4bc210

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-4bc210

openshift-ingress

deployment-controller

e2e-worker-single-main-4bc210-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-single-main-4bc210-openshift-default-8689fcffdf from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-single-main-4bc210-openshift-default-8689fcffdf

SuccessfulCreate

Created pod: e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

openshift-ingress

multus

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

AddedInterface

Add eth0 [10.134.0.194/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-4bc210

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-4bc210: serviceaccount "payload-processing-e2e-worker-single-main-4bc210" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-4bc210

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-4bc210: serviceaccount "payload-processing-e2e-worker-single-main-4bc210" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

AddedInterface

Add eth0 [10.132.0.168/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2

AddedInterface

Add eth0 [10.132.0.169/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2

Started

Container started

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Unhealthy

Readiness probe failed: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

ProbeError

Readiness probe error: Get "http://10.134.0.194:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Unhealthy

Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

ProbeError

Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-4bc210-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-4bc210-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7775st

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cfr2qq2

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-4bc210: serviceaccount "payload-processing-e2e-worker-single-main-4bc210" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-4bc210

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-4bc210-554c7ccd79 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-4bc210: serviceaccount "payload-processing-e2e-worker-single-main-4bc210" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-4bc210

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cf

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z

AddedInterface

Add eth0 [10.132.0.173/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb

AddedInterface

Add eth0 [10.132.0.172/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-4bc210-f5fbf99cftl55z

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-4bc210-554c7j7flb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-single-main-4bc210-openshift-default-8689fcffdfv8k9x

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zvq6k

Unhealthy

Readiness probe failed: Get "http://10.134.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zvq6k

ProbeError

Readiness probe error: Get "http://10.134.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

Unhealthy

Readiness probe failed: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-zdbqj

ProbeError

Readiness probe error: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-89nq9

Unhealthy

Readiness probe failed: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55fbd65575-89nq9

ProbeError

Readiness probe error: Get "http://10.134.0.117:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body: